Ultralytics
integrations/tflite/ #8856
giscus[bot]
bot
Replies: 16 comments 61 replies
-
|
Mobile SSD models are expected to have exactly 4 outputs, found 1, I get this error trying to integrate on Android, i added metadata for input. How do I suppose to handle this, and can you provide example? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like you're encountering an issue with the expected outputs for your Mobile SSD model on Android. This typically happens when the model's output layer configuration doesn't match what the Android integration expects. For YOLOv8 models, including when exported to TFLite, the model should output more than just one array. Specifically, you'd expect outputs for bounding boxes, objectness scores, and class probabilities at the very least. Here's a quick example on how to export your YOLOv8 model to TFLite, which should be compatible with Android: from ultralytics import YOLO
# Load your YOLOv8 model
model = YOLO('path/to/your/model.pt')
# Export to TFLite
model.export(format='tflite')After exporting, ensure your Android application is set up to handle all the outputs from the TFLite model. You might need to adjust your Android code to parse these outputs correctly. For more detailed guidance on deploying YOLOv8 models to Android, including handling model outputs, check out our TFLite integration guide. It covers everything from exporting your model to deploying it on edge devices, including Android. Hope this helps! Let us know if you have any more questions. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
"TensorFlow SavedModel: export failure ❌ 49.4s: generic_type: cannot initialize type "StatusCode": an object with that name is already defined" Im having this error occur when I try to export my yolov8 weights to tflite format. Any idea how to solve this? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! It seems like there's a conflict with an already defined object A potential quick fix could be to try isolating your export environment. You can create a fresh virtual environment specifically for exporting your model. Here's how you can do that: # Create a new virtual environment
python3 -m venv yolov8-export-env
# Activate the environment (Unix/macOS)
source yolov8-export-env/bin/activate
# (Windows)
.\yolov8-export-env\Scripts\Activate
# Install the specific version of TensorFlow you're working with
pip install tensorflow==your.version.here ultralytics
# Attempt the export again
yolo export model=path/to/your/model format=tfliteIf the issue persists, it might be helpful to ensure there's no conflicting package or to see if there's an updated version of TensorFlow that addresses this error. Checking TensorFlow's GitHub issues for similar cases might shed more light on this. 🤔 Hope this helps! Let us know how it goes. |
Beta Was this translation helpful? Give feedback.
-
|
I am trying to export the yolov8 default model (yolov8n.pt) into TFlite but after inputing the code Load a modelmodel = YOLO('yolov8n.pt') # build a new model from YAML Train the modelmodel.export(format='tflite') I get the error message ERROR: The trace log is below. What you should do instead is wrap ERROR: input_onnx_file_path: yolov8n.onnx |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It seems like you're facing some challenges while trying to export the YOLOv8 model to TFLite format. The error message you encountered often occurs when there's an issue with the conversion process, particularly with ONNX to TensorFlow transformations. To get around this issue, you might want to first export your model to ONNX and then use TensorFlow's tools for converting ONNX to TFLite. Here's a step-by-step way you can try:
Remember, the conversion process can be tricky due to the complex architectures of models like YOLOv8. It's possible you may encounter issues specific to certain layers or operations that aren't directly convertible or supported in TFLite. For more detailed guidance, check out the Ultralytics guide on exporting models to TFLite, which provides valuable insights and best practices. If you run into further issues or specific errors during the conversion process, feel free to share them here. The community and the Ultralytics team are here to help! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the help. Fortunatly I have discovered that the package "tflite-support" only works for python 3.8 when I was using python 3.11. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😃 Great job on getting the TFLite export to work by switching to Python 3.8! It's interesting how specific versions can make a difference, and it's a good reminder to always check compatibility. If you—or anyone else—runs into similar issues in the future, this insight about the Python version could be a super helpful tip. Additionally, it's always a good idea to check the requirements for the latest updates on supported packages and Python versions to ensure smooth sailing. Thanks for sharing your solution with the community! Should you have any further questions or run into other challenges, feel free to reach out. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hey,I met a problem，these my conda list absl-py 2.1.0 When I trans the .pt to .tflite,it happened this PyTorch: starting from 'E:\Lady_YOLO\YAML\yolov8n.pt' with input shape (1, 3, 160, 128) BCHW and output shape(s) (1, 84, 420) (6.2 MB) This is my code,and the model is official from ultralytics import YOLO |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🙋♂️ It looks like you encountered an error while converting a YOLOv8 model to TFLite format. The error message from ultralytics import YOLO
model = YOLO("E:/Lady_YOLO/YAML/yolov8n.pt")
model.export(format='tflite', imgsz=(160,128)) # Note: imgsz as a tuple (width, height)Try switching |
Beta Was this translation helpful? Give feedback.
-
|
Hey!Thanks for your help,and my question is totally sloved.It's our honor to have the developers like you! |
Beta Was this translation helpful? Give feedback.
-
|
@asongdashi hey! 🎉 We're thrilled to hear your issue was resolved and appreciate your kind words! Our developers are genuinely passionate about supporting users like you. Should you have any more questions or need further assistance, don't hesitate to reach out. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
I am trying to run a yolov8 model on tflite in c++ on a linux arm v8l. May I know if tflite can support this and the version that supports it? Would appreciate any help |
Beta Was this translation helpful? Give feedback.
-
|
@Thunderzen yes, TensorFlow Lite (TFLite) does support running on a Linux ARM v8l architecture, which should suit your requirements for deploying a YOLOv8 model in C++. You can export your trained YOLOv8 model to the TFLite format using the Ultralytics framework and then deploy it in your ARM-based environment. Here's a brief example on how to export your model to TFLite using Python: from ultralytics import YOLO
# Load your trained YOLOv8 model
model = YOLO('path/to/your/model.pt')
# Export the model to TFLite format
model.export(format='tflite')This will generate a |
Beta Was this translation helpful? Give feedback.
-
|
Hi, i trained a model yolov8 with custom dataset containing 26 classess, but when i convert the model to tflite i noticed that it gives as output [1,30,8400] and this is what caused me errors when using my model with flutter. the error |
Beta Was this translation helpful? Give feedback.
-
|
@bachir172 hi, The output shape mismatch you're observing seems related to the output configuration expected by your Flutter application. Your TFLite model produces an output of [1, 30, 8400], which likely includes a placeholder for the maximum number of detected classes per anchor point (plus additional detection-related metrics). The error arises because the Flutter app expects [1, 26] as the shape. This discrepancy suggests that the TensorFlow Lite conversion or the application's handling of the model output needs adjustment. Here's a quick suggestion to help resolve this:
Also, ensure the application correctly handles this output shape both in layout (like reading the correct indices) and in datatype expectations. If issues persist, you may need to delve deeper into how outputs are structured post-conversion and ensure everything from the model export to the application's handling aligns correctly. Hope this helps! Let me know how it goes. 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hello |
Beta Was this translation helpful? Give feedback.
-
|
ThankYou Very much The goal is to be able to use the ultralytics object counter for the tflite int8 file |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you for your kind words and for sharing your goal with us. We're excited to help you use the Ultralytics object counter with a TFLite INT8 model. Let's get you set up! Exporting to TFLite INT8First, ensure you have the latest versions of pip install --upgrade torch ultralyticsNext, you can export your YOLOv8 model to TFLite INT8 format using the following code: from ultralytics import YOLO
# Load the YOLOv8 model
model = YOLO("yolov8n.pt")
# Export the model to TFLite INT8 format
model.export(format="tflite", int8=True) # creates 'yolov8n_int8.tflite'Running Inference with the Exported TFLite INT8 ModelAfter exporting, you can load and run inference with the TFLite model: from ultralytics import YOLO
# Load the exported TFLite INT8 model
tflite_model = YOLO("yolov8n_int8.tflite")
# Run inference
results = tflite_model("https://ultralytics.com/images/bus.jpg")
# Display results
results.show()Integrating with MediaPipeTo leverage MediaPipe for inference, you can use the following example: import cv2
import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
def run(model: str, video_path: str) -> None:
cap = cv2.VideoCapture(video_path)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
base_options = vision.ObjectDetectorOptions(
base_options=python.BaseOptions(model_asset_path=model),
max_results=3,
score_threshold=0.6
)
detector = vision.ObjectDetector.create_from_options(base_options)
while cap.isOpened():
success, image = cap.read()
if not success:
break
rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
input_tensor = mp.Image(image_format=mp.ImageFormat.SRGB, data=rgb_image)
detection_result = detector.detect(input_tensor)
image = visualize(image, detection_result)
if cv2.waitKey(1) == 27:
break
cv2.imshow('object_detector', image)
cap.release()
cv2.destroyAllWindows()
def visualize(image, detection_result):
for detection in detection_result.detections:
bbox = detection.bounding_box
start_point = (int(bbox.origin_x), int(bbox.origin_y))
end_point = (int(bbox.origin_x + bbox.width), int(bbox.origin_y + bbox.height))
cv2.rectangle(image, start_point, end_point, (0, 255, 0), 2)
label = detection.categories[0].category_name
score = detection.categories[0].score
label_text = f"{label}: {score:.2f}"
cv2.putText(image, label_text, (int(bbox.origin_x), int(bbox.origin_y) - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)
return image
run('yolov8n_int8.tflite', 'video.mp4')This example demonstrates how you can use MediaPipe to run inference on a video file using a TFLite INT8 model. For more detailed guidance, please refer to our TFLite integration guide. We appreciate your patience and support as we continue to enhance our integration capabilities. If you encounter any further issues or have additional suggestions, please feel free to reach out. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
after run export: Automatic generation of each OP name started ======================================== Model loaded ======================================================================== Model conversion started ============================================================ ERROR: The trace log is below. Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_5/Add}} = AddV2[T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,16,160,32], [1,16,160,16]. Call arguments received by layer "tf.math.add_5" (type TFOpLambda): ERROR: input_onnx_file_path: yolov8n.onnx |
Beta Was this translation helpful? Give feedback.
-
|
after inference with mymodel.tflite: WARNING |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! Thank you for sharing the details of the issue you're encountering. Let's work together to resolve this. 😊 Issue AnalysisThe error you're seeing indicates a problem with the metadata parsing in the TFLite model. Specifically, it seems like the model's metadata might not be correctly formatted or is missing, leading to a Steps to Resolve
Example Code for Object CountingTo integrate the object counter with the TFLite INT8 model, you can use the following code: import cv2
from ultralytics import YOLO, solutions
# Load the exported TFLite INT8 model
tflite_model = YOLO("yolov8n_int8.tflite", task='detect')
# Open video file
cap = cv2.VideoCapture("video.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))
# Define region points
region_points = [(20, 400), (1080, 404), (1080, 360), (20, 360)]
# Video writer
video_writer = cv2.VideoWriter("object_counting_output.avi", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))
# Init Object Counter
counter = solutions.ObjectCounter(
view_img=True,
reg_pts=region_points,
classes_names=tflite_model.names,
draw_tracks=True,
line_thickness=2,
)
while cap.isOpened():
success, im0 = cap.read()
if not success:
print("Video frame is empty or video processing has been successfully completed.")
break
tracks = tflite_model.track(im0, persist=True, show=False)
im0 = counter.start_counting(im0, tracks)
video_writer.write(im0)
cap.release()
video_writer.release()
cv2.destroyAllWindows()Additional ResourcesFor more details on exporting and deploying YOLOv8 models to TFLite, please refer to our TFLite integration guide. If you continue to experience issues, please provide a minimum reproducible example so we can investigate further. You can find more information on creating a reproducible example here. Thank you for your patience and support. We're here to help! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, The following is the content of my requirements.txt: and am using the following code for the conversion: Load the YOLOv8 modelmodel = YOLO("current_best4.pt") Export the model to TFLite formatmodel.export(format="tflite", int8=True, data='./datasets/data.yaml') # creates 'yolov8n_float32.tflite' Load the exported TFLite modeltflite_model = YOLO("yolov8n_int8.tflite") Run inferenceresults = tflite_model("./frame_365.jpg") results.show() Any help will be greatly appreciated |
Beta Was this translation helpful? Give feedback.
-
|
This is the full log when I run the code sample, PyTorch: starting from 'current_best4.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 7, 8400) (6.0 MB) TensorFlow SavedModel: starting export with tensorflow 2.13.0... ONNX: starting export with onnx 1.15.0 opset 17... Automatic generation of each OP name started ======================================== Model loaded ======================================================================== Model conversion started ============================================================ ERROR: input_onnx_file_path: current_best4.onnx |
Beta Was this translation helpful? Give feedback.
-
|
Hi there, Thank you for providing the detailed log and requirements. It looks like you're encountering an issue related to the TensorFlow operations during the TFLite conversion process. This error often arises due to the incompatibility of certain TensorFlow operations with the TFLite conversion process. Here are a few steps to help resolve this issue:
Here is an example of how you might modify your code to handle some of these suggestions: from ultralytics import YOLO
# Load the YOLOv8 model
model = YOLO("current_best4.pt")
# Export the model to TFLite format with static input shape
model.export(format="tflite", int8=True, data='./datasets/data.yaml', imgsz=640) # creates 'yolov8n_float32.tflite'
# Load the exported TFLite model
tflite_model = YOLO("yolov8n_int8.tflite")
# Run inference
results = tflite_model("./frame_365.jpg")
results.show()If the issue persists, you might need to delve deeper into the specific TensorFlow operations causing the problem. For more detailed guidance, please refer to the TensorFlow Lite documentation and the Ultralytics TFLite export guide. Feel free to reach out if you have any further questions or need additional assistance. Happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for the quick response, let me try this solution. |
Beta Was this translation helpful? Give feedback.
-
|
I was able to run the code sample in Google Colab and everything worked out fine. I appreciate the help. |
Beta Was this translation helpful? Give feedback.
-
|
@bigg-S hi there, I'm glad to hear that everything worked out well for you in Google Colab! 🎉 If you have any further questions or run into any other issues, feel free to reach out. We're here to help! For more detailed information on exporting YOLOv8 models to TFLite and deploying them on various platforms, you can refer to our TFLite export guide. This guide covers everything from installation to deployment, ensuring you have all the resources you need for a smooth experience. Happy coding and best of luck with your project! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
import cv2 cap.release() For this code I'm getting this error TensorFlow SavedModel: export failure ❌ 47.0s: No module named 'tensorflow_lite_support' |
Beta Was this translation helpful? Give feedback.
-
|
@aastha0612 hi there! Thank you for reaching out. It looks like you're encountering an issue with exporting your YOLOv8 model to TFLite due to a missing module, To resolve this, you need to install the pip install tensorflow_lite_supportAfter installing the package, try running your export command again. Here’s a quick example to ensure everything is set up correctly: from ultralytics import YOLO
# Load the YOLOv8 model
model = YOLO("yolov8n.pt")
# Export the model to TFLite format
model.export(format="tflite", imgsz=320, half=True)Additionally, please ensure you are using the latest version of the Ultralytics package by running: pip install --upgrade ultralyticsIf the issue persists, please provide a minimum reproducible example so we can further investigate. You can find more details on how to create one here: Minimum Reproducible Example. For more detailed instructions on exporting to TFLite, you can refer to our TFLite Export Guide. Feel free to reach out if you have any more questions! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Hello |
Beta Was this translation helpful? Give feedback.
-
|
@Josephgvvvn hello, Thank you for your detailed feedback and observations. It's indeed crucial to consider the operating system when addressing issues related to TFLite, as support can vary. For Windows users, installing Regarding your question on model and image sizes for TFLite and ONNX formats:
Here’s a quick example to clarify: from ultralytics import YOLO
# Load the YOLOv8 model
model = YOLO("yolov8n.pt")
# Export the model to TFLite format with square dimensions
model.export(format="tflite", imgsz=640, half=True)For more details on exporting and using TFLite models, you can refer to our TFLite Export Guide. Feel free to reach out if you have any more questions! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Hi! I trying to convert my model .pt to .tflite but i get this error: Ultralytics YOLOv8.2.73 🚀 Python-3.8.19 torch-2.1.2+cpu CPU (11th Gen Intel Core(TM) i5-1145G7 2.60GHz) PyTorch: starting from 'runs\classify\train\weights\last.pt' with input shape (1, 3, 64, 64) BCHW and output shape(s) (1, 4) (2.8 MB) TensorFlow SavedModel: starting export with tensorflow 2.13.0... I have already tried to install the requirements indicated there but I get a compatibility error. Can you help me please? |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for sharing your solution. Downgrading ONNX from 1.16.2 to 1.15.0 can indeed resolve compatibility issues during the TFLite export process. If you encounter any further issues or need additional assistance, please refer to our TFLite export documentation for more detailed guidance. |
Beta Was this translation helpful? Give feedback.
-
|
Thank u for your solutions. I trying to downgrade onnx version to 1.62.1 and it worked for me! I still thank you for the support. |
Beta Was this translation helpful? Give feedback.
-
|
I'm glad to hear that downgrading ONNX to 1.15.0 worked for you! If you encounter any further issues or need additional guidance, feel free to refer to our TFLite export documentation. Thank you for your patience and for sharing your solution. |
Beta Was this translation helpful? Give feedback.
-
|
I had the same exact problem, I think it is due to some local python settings. When i tried to do the exact same command on a google-colab project, it all went smoothly. (I am referring to the auto-download failures here) And when those worked properly, it all worked properly afterwards. |
Beta Was this translation helpful? Give feedback.
-
|
Using Google Colab can help bypass local environment issues, as it provides a pre-configured setup that often resolves dependency conflicts. If you need further assistance, feel free to explore our documentation for more insights. |
Beta Was this translation helpful? Give feedback.
-
|
i trained a custom model of yolov8segm converted into tflite but when i imported it on my android studio project, it says that it's an invalid model. what should i do? the model works properly in colab and segments the object in the image |
Beta Was this translation helpful? Give feedback.
-
|
i also tried a different version of android studio but it doesnt work. i managed to make it work when i tried non-seg model but i need to use segmentation |
Beta Was this translation helpful? Give feedback.
-
|
It seems like the issue might be related to the segmentation model's compatibility with Android Studio. Ensure that the TFLite model is correctly exported and that your Android project is set up to handle segmentation tasks. You might also want to check if the TensorFlow Lite version in your Android project matches the version used during export. For further guidance, refer to the TFLite documentation. |
Beta Was this translation helpful? Give feedback.
-
|
sorry i might be asking too much but i trained model using |
Beta Was this translation helpful? Give feedback.
-
|
@edjjff it sounds like there might be an issue with the TFLite model compatibility in your Android project. Ensure that the TensorFlow Lite version in your Android Studio matches the version used during export. Also, verify that your project setup supports segmentation tasks. For more detailed guidance, you can refer to the TFLite documentation. |
Beta Was this translation helpful? Give feedback.
-
|
When exporting the pre-trained model to TFLite: My export is crashing due to the process trying to allocate 26GB of RAM memory ( exceeding available limit) As I have understood it, the data-parameter in the export is to verify the quantization (int8-param) When using the default coco8.yaml file, the process in complaining that the verification is not optimal (It wants atleast 1000 images, but coco8 provides just 4). So to enahnce this, I thought: why not use the entire coco-dataset? The question being: Am I correct to use the entire coco-dataset if i want the quantization to become better, or should i just stick to the default. And if the answer is: yes, use the entire coco-set, how do I overcome the huge allocation of RAM for that? is there a dataset with say 1000 images, instead of the 120K+ images of the entire coco-set? |
Beta Was this translation helpful? Give feedback.
-
|
@Olofasenius using the entire COCO dataset can improve quantization, but it may lead to high memory usage. Consider using a smaller dataset with around 1000 images for a balance between performance and resource usage. Alternatively, you can try reducing the batch size or using a machine with more RAM. For more details, please refer to the Ultralytics documentation on exporting. |
Beta Was this translation helpful? Give feedback.
-
|
Issue with Incorrect Predictions from Quantized YOLOv8m-obb Model (TFLite) in TensorFlow Framework I have exported my YOLOv8m-obb model to TFLite format with INT8 quantization enabled, using an image size of 640x640 and a data.yaml for my dataset. When I use the quantized model for inference with the Ultralytics framework (Oriented Bounding Boxes), the predictions are correct. However, when I use the same model in the TensorFlow framework, I encounter several issues with the output: I suspect there might be an issue with how the quantization parameters (scale and zero-point) are being applied in TensorFlow, or possibly with the way I'm handling the model's output or the way I exported the model. I would appreciate guidance on how to correctly handle the quantized model in TensorFlow and resolve the issues with incorrect predictions. |
Beta Was this translation helpful? Give feedback.
-
|
TFLite Model Export and Implementation Code yolov8m-obb I exported my YOLO model to TFLite format using the following CLI command: yolo export obb model="best_obb.pt" format="tflite" imgsz=640 int8=True data="card_detected/data.yaml" Below is the Python code I plan to use with the exported model:
The exported TFLite model is loaded and initialized as follows: model_path = "path/to/model.tflite"
Steps for preprocessing the input image: def preprocess_image(image_path, input_shape, quant_params): prepared_image = preprocess_image(image_path, (640, 640), quantization_params)
Steps to run inference on the preprocessed image: input_index = input_details[0]['index']
Steps for processing the output predictions: output_index = output_details[0]['index'] for pred in predictions: |
Beta Was this translation helpful? Give feedback.
-
|
It appears you've implemented the correct steps for TFLite export and inference. However, the issues with confidence scores and prediction outputs in TensorFlow may arise from incorrect handling of quantization parameters or preprocessing. Double-check the input data scaling (scale, zero_point) and ensure the preprocessed image matches the model’s input expectations. If the issue persists, consider testing the model using the original Ultralytics framework to isolate whether TensorFlow-specific handling is causing the discrepancies. For additional guidance, refer to the Ultralytics TFLite Export Documentation. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your feedback! Unfortunately, the issue still persists. I tested the model directly on the Ultralytics framework, and it works perfectly, making predictions without any problems. This confirms that the model is correctly trained and exported. Regarding my implementation, I thoroughly checked the following steps: Despite these checks, the predictions obtained through my implementation remain inconsistent or incorrect. I can’t pinpoint the root cause of this issue. Do you have any additional suggestions or insights to help me resolve this? Thank you in advance for your assistance! |
Beta Was this translation helpful? Give feedback.
-
|
It seems your implementation aligns with the TFLite specifications, but the issue might be related to how TensorFlow Lite's interpreter processes the quantized model. I recommend verifying the order and accuracy of the input tensor shape and quantization scale/zero-point during preprocessing. Additionally, ensure that the output tensor dimensions and data types match the exported model's specifications. If the issue persists, consult the Ultralytics TFLite guide for debugging tips or consider testing on a different TensorFlow Lite runtime version. |
Beta Was this translation helpful? Give feedback.
-
|
Hello @rihablahouar, I came across this issue you raised and wanted to check if you were able to find a solution. I am currently attempting to run inference for object detection using a YOLOv11 Nano model that I exported to TensorFlow Lite, and I’m encountering the same challenges you described. After some investigation, I suspect there might be differences between how the YOLO framework handles the process and how TensorFlow Lite operates. As you mentioned, I can run the inference using the YOLO framework and I can confirm it works fine! If you’ve managed to resolve this or have any insights, I’d greatly appreciate your guidance. Thank you in advance! |
Beta Was this translation helpful? Give feedback.
-
|
Issue with Incorrect Predictions from Quantized YOLOv8m-obb Model (TFLite) in TensorFlow Framework I have exported my YOLOv8m-obb model to TFLite format with INT8 quantization enabled, using an image size of 640x640 and a data.yaml for my dataset. When I use the quantized model for inference with the Ultralytics framework (Oriented Bounding Boxes), the predictions are correct. However, when I use the same model in the TensorFlow framework, I encounter several issues with the output: I suspect there might be an issue with how the quantization parameters (scale and zero-point) are being applied in TensorFlow, or possibly with the way I'm handling the model's output or the way I exported the model. I would appreciate guidance on how to correctly handle the quantized model in TensorFlow and resolve the issues with incorrect predictions. |
Beta Was this translation helpful? Give feedback.
-
|
@rihablahouar the issues you're experiencing with TensorFlow likely stem from differences in how TFLite handles quantization parameters and outputs compared to the Ultralytics framework. Ensure you're correctly interpreting the TFLite model's output by referencing its metadata. Additionally, verify the quantization parameters (scale and zero-point) are correctly applied during inference. For more details on TFLite usage and troubleshooting, refer to the TFLite export guide. |
Beta Was this translation helpful? Give feedback.
-
|
TFLite Model Export and Implementation Code, yolov8m-obb I exported my YOLO model to TFLite format using the following CLI command: yolo export obb model="best_obb.pt" format="tflite" imgsz=640 int8=True data="card_detected/data.yaml" Below is the Python code I plan to use with the exported model:
The exported TFLite model is loaded and initialized as follows: model_path = "path/to/model.tflite"
Steps for preprocessing the input image: def preprocess_image(image_path, input_shape, quant_params): prepared_image = preprocess_image(image_path, (640, 640), quantization_params)
Steps to run inference on the preprocessed image: input_index = input_details[0]['index']
Steps for processing the output predictions: output_index = output_details[0]['index'] for pred in predictions: |
Beta Was this translation helpful? Give feedback.
-
|
Your implementation looks solid, but the issues you're encountering may arise from input preprocessing or output interpretation. Ensure the quantization parameters (scale and zero_point) are consistently applied during preprocessing and postprocessing. For oriented bounding boxes (OBB), verify that the output tensor format aligns with the model's specifications, and refer to the TFLite export guide for additional insights. Let us know if the problem persists! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your feedback! Unfortunately, the issue still persists. I tested the model directly on the Ultralytics framework, and it works perfectly, making predictions without any problems. This confirms that the model is correctly trained and exported. Regarding my implementation, I thoroughly checked the following steps: Despite these checks, the predictions obtained through my implementation remain inconsistent or incorrect. I can’t pinpoint the root cause of this issue. Do you have any additional suggestions or insights to help me resolve this? Thank you in advance for your assistance! |
Beta Was this translation helpful? Give feedback.
-
|
It seems like you’ve thoroughly followed the preprocessing and dequantization steps. However, given the inconsistency in TensorFlow outputs, I recommend verifying the output tensor shapes and their interpretation against the model's metadata. Additionally, check if the TFLite interpreter is correctly initialized for INT8 quantization. For further guidance, refer to the TFLite export guide. Let us know if the issue persists! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your feedback! Unfortunately, the issue still persists. I
tested the model directly on the Ultralytics framework, and it works
perfectly, making predictions without any problems. This confirms that the
model is correctly trained and exported.
Regarding my implementation, I thoroughly checked the following steps:
1. *Image preprocessing*: Resizing and normalization are done correctly
using the formula quantized=scalereal+zero_point.
2. *Dequantization of outputs*: I applied the formula dequantized=(
quantized−zero_point)×scale, which seems consistent with the TFLite
specifications.
Despite these checks, the predictions obtained through my implementation
remain inconsistent or incorrect. I can’t pinpoint the root cause of this
issue. Do you have any additional suggestions or insights to help me
resolve this?
Thank you in advance for your assistance!
Le mer. 25 déc. 2024 à 20:12, Glenn Jocher ***@***.***> a
écrit :
… Your implementation looks solid, but the issues you're encountering may
arise from input preprocessing or output interpretation. Ensure the
quantization parameters (scale and zero_point) are consistently applied
during preprocessing and postprocessing. For oriented bounding boxes (OBB),
verify that the output tensor format aligns with the model's
specifications, and refer to the TFLite export guide
<https://docs.ultralytics.com/integrations/tflite/> for additional
insights. Let us know if the problem persists!
—
Reply to this email directly, view it on GitHub
<#8856 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ASDVJFIICL5CHUGNJGLU5JL2HL7TBAVCNFSM6AAAAABEQM3SIKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRWGQ4DCNA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
@rihablahouar it seems your implementation is thorough, but inconsistencies may arise from subtle differences in preprocessing or postprocessing. Verify that the input tensor dimensions, quantization scale, and zero-point match the exported model's specifications. Additionally, ensure the TFLite Interpreter is correctly configured for inference. For further guidance, refer to the TFLite integration guide. Let us know if this helps! |
Beta Was this translation helpful? Give feedback.
-
|
The Pytorch ultralytics framework works and returns the correct prediction. However, if I do it in the TensorFlow Lite framework, the prediction results are wrong for the same image and the same pre-processing. |
Beta Was this translation helpful? Give feedback.
-
|
It seems the issue arises from inconsistencies between the PyTorch and TensorFlow Lite frameworks. Ensure that the TFLite input tensor dimensions, preprocessing (e.g., resizing, normalization), and output dequantization steps align perfectly with the exported model's specifications. For detailed guidance, review the TFLite integration guide. Let us know if further clarification is needed! |
Beta Was this translation helpful? Give feedback.
-
|
hi there @glenn-jocher I am trying to convert the the yolov10 model to int8 with tflite format. but same error is occurring again and again. I've tried the solutions described in this thread but nothing seems to work. I am using this for conversion: ` from ultralytics import YOLO model = YOLO("current_best4.pt") model.export(format="tflite", int8=True, data='./datasets/data.yaml', imgsz=640) # creates 'yolov8n_float32.tflite' tflite_model = YOLO("yolov8n_int8.tflite") results = tflite_model("./frame_365.jpg") results.show() ` and the error i am receiving is this:
i I have approx 5k images for calibration, so thats not the issue, all the packages are upto date as well. can you please suggest what solution should i opt for? |
Beta Was this translation helpful? Give feedback.
-
|
@saleha-fatima the error appears to be related to shape mismatches during ONNX to TFLite conversion. Please ensure that your model, dataset, and TensorFlow versions are fully compatible. Refer to the onnx2tf parameter replacement guide for addressing this issue. Additionally, try exporting with |
Beta Was this translation helpful? Give feedback.
-
|
I've tried with torch==2.0.1 I have alse tried the suggestions mentioned in error msg but nothing seems to work! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing the details. This appears to be a compatibility issue between YOLOv10's architecture and the ONNX->TFLite conversion path. Please try: 1. Upgrade |
Beta Was this translation helpful? Give feedback.
-
integrations/tflite/
Explore how to improve your Ultralytics YOLOv8 model's performance and interoperability using the TFLite export format suitable for edge computing environments.
https://docs.ultralytics.com/integrations/tflite/
Beta Was this translation helpful? Give feedback.
All reactions