Custom tflite model doesn't detect anything

[fransay]

Hi peeps,

I have been trying to train a custom model for object detection using this package, i.e flutter_tflite, but have found no success. custom models when ingested into the example app does not detect anything. Below are the approaches I have taken so far:

1. Training with tflite_model_maker
   model_spec = 'efficientDET_lite0'

2. Tensorflow 2 Object Detection API
   model_spec = 'ssd_mobilenet_v1/v2' && 'ssd_resnet50'

3. Ultrayltics
   model_spec = yolo v8, v7, v5 -> output to a tflite.

None of the above has led to success, I am not able to perform detection on my honor x6a using the example app found here:https://github.com/tensorflow/flutter-tflite/tree/main/example/live_object_detection_ssd_mobilenet

Any help on instructions to train a custom working model for live detection use cases will be meet with a ton of gratitude and appreciation. Looking forward for help.
Thanks.

[GustavoSantiago113]

On the yolov8 model I have used the flutter vision package: https://pub.dev/packages/flutter_vision. I hope it works for you too

[fransay]

@GustavoSantiago113 , thank you for your reply! Flutter vision is definitely a very great package by all standards, only limitation is the IOS support is not yet ready. Let's hope Yuri or any other contributor has sometime to enable support in that direction. It stand tall among the best plugins for object detection in flutter.
Again thank you very much for the suggestion.

[darwinOne]

Hi @fransay
Did you find the solution?
I had the same problem: not detecting anything with the yolov8 custom model.

[einsitang]

@fransay @darwinOne hello
same problem too , I got two yolov8 model(tflite format export) use on tflite_flutter , one is ok , by another is alway not detect , so I very sure the preprocess and postprocess function is correct, and I have checked that both models perform well in tensorflow python.

so did you find the solution or reason of the problem

[fransay]

@einsitang @darwinOne , ultralytics is promising to deliver a stable tool for the flutter ecosystem. https://github.com/ultralytics/yolo-flutter-app. It is still a work in progress, but much better and does support yolo model builds.
You can also see one from a bright korean dev, does support both pytorch and coreml. Here is it: https://github.com/spring98/flutter-yolo-realtime-plugin
Hope these resources help.

[einsitang]

Hope these resources help.

thank you, I try yolo-flutter-app before , but the output result has a big deviation, maybe I used the wrong way, so I need to adjust myself, that is why I write my own logic processing. In addition, I hope that if I change or add other models in the future, I can handle it myself, rather than just using yolo

[fransay]

@einsitang, you mean deviation of predictions from truth ?

[einsitang]

@einsitang, you mean deviation of predictions from truth ?

yes ， I’am very sure the preprocess and postprocess is correct ， because one model is output detection is right，but another is empty（they all output detections on python platform with same image）

[fransay]

It is an interesting issue you found, not sure if I've seen any issue similar to what you are encountering. For me, I found out that my inference speed on my honor x6a https://www.gsmarena.com/honor_x6a-12442.php, was quite low. Definitely there is a lot of work to be done on improving the package. I am yet to have a great experience with any of the machine learning packages, I have come across.
I don't know if I have enough time, I'd love to write my own package from scratch in Kotlin, and call utilities from flutter.

[einsitang]

I have been trying to train a custom model for object detection using this package, i.e flutter_tflite, but have found no success. custom models when ingested into the example app does not detect anything. Below are the approaches I have taken so far

so , your model detect work now ? seem your problem is about inference speed , that is performance problem, For performance optimization, the solution I can find so far is to quantify the model.

[fransay]

Custom detection doesn't work with flutter-tflite. Inference is also an add-on issue, ready to be tackled.

[fransay]

What techniques of quantification do you use ?

[einsitang]

I haven't done the quantification work yet , because the model can't detect and I have no idea how to fix

[einsitang]

I just solved the second model detect problem , if your problem is about yolov8 model , you can show the code , maybe I can help you

[fransay]

@einsitang I sent you a private mail ✉️ , can we continue from there ?

[einsitang]

@einsitang I sent you a private mail ✉️ , can we continue from there ?

You can email me, but I am not sure that I can solve the problem you encounter, only try

[fransay]

Sure, no problem!

[einsitang]

ok,you can find the email in my profile

[fransay]

ok,you can find the email in my profile

Sent. Thank you.
Try and check spam box, since I did send one yesterday, but seems you didn't get to see it.

[einsitang]

ok,you can find the email in my profile

Sent. Thank you. Try and check spam box, since I did send one yesterday, but seems you didn't get to see it.

I just found out you emailed me, and did found in spam box 😂