guides/preprocessing_annotated_data/

[giscus[bot]]

guides/preprocessing_annotated_data/

Learn essential data preprocessing techniques for annotated computer vision data, including resizing, normalizing, augmenting, and splitting datasets for optimal model training.

https://docs.ultralytics.com/guides/preprocessing_annotated_data/

[astronitya]

Hello,
I have a question related to training image size, I work with astronomical images, and the image size I have are in size of 2kx4k, can I train a custom model with this image size?

[glenn-jocher]

If resizing is causing a loss of detail in your astronomical images, consider using image tiling to divide your larger images into smaller segments without reducing resolution. This way, you can maintain detail and train the model on these high-resolution fragments. Adjust your dataset configuration to accommodate this approach. For more details, refer to the Data Preprocessing Techniques guide.

[valschmidt]

@glenn-jocher, could you comment more on best practices for both training and inference with regard to image size? We did a test recently in which we evaluated the performance of a model with the imgsz parameter set to varying values, both larger and smaller than the native image size. We found that the model performs much worse (almost by half) when imgsz is not matched to the actual size of the image. We thought about tiling too, as you have suggested, but this comes at a computational price and our application is real-time. This suggests that to properly evaluate a model, and indeed deploy it, the value of imgsz should not be set to a constant value, but should be automatically varied to match the input images. Is this currently possible with any of the CLI or canned tools provided by ultralytics?

The sensitivity of the performance of the model to the imgsz value during inference then gives me pause for thought regarding how it should be set during training. It seems it too should match the native image size, and anything else is unnecessarily throwing away information. But is it possible using the ultralytics tools to change it on the fly during training? Do I understand this correctly? I'd be curious to hear your recommendations here on the imgsz parameter during training.

I guess I am surprised that, given the sensitivity of model performance to this parameter, it is not more widely discussed nor central to the discussions of model performance. I feel like I am missing something fundamental.

[glenn-jocher]

Thank you for your detailed observations. Currently, YOLO's imgsz parameter does not dynamically adjust to match varying input sizes during training or inference. However, you can explore image tiling for detailed features or use the rect parameter for inference to handle non-uniform image dimensions more efficiently. For training, setting imgsz close to the native resolution can preserve more detail, but it must balance with computational constraints. For more on preprocessing and resizing techniques, refer to Data Preprocessing Techniques.

[valschmidt]

Thanks for your reply. Fascinating.

I am interested to learn more about the change in Yolo from early versions in which the model input was fixed and later versions in which the imgsz parameter allowed the model input size to change dynamically. I've tried to look up the seminal papers that introduced each of the versions but I can't seem to find a discussion that directly addresses the development that allowed for a model that can adapt to the image size.

Would you happen to know in what version of Yolo the imgsz parameter was introduced?
Could you recommend a paper that discusses this major change in the way Yolo is architected?

[glenn-jocher]

The introduction of the imgsz parameter and dynamic input resizing was first seen in YOLOv5, developed by Ultralytics. This feature wasn't explicitly discussed in a single seminal paper but emerged as part of the iterative advancements in YOLO's architecture. For a historical perspective, you can explore the YOLOv5 repository and its documentation.

[UltralyticsAssistant]

👋 Hello, thank you for your interest in Ultralytics 🚀! It looks like you're diving into data preprocessing techniques for annotated computer vision data. That's a great step for optimizing your model training! Make sure to explore the guides as they provide essential information on resizing, normalizing, augmenting, and splitting datasets.

If you encounter any issues or have questions while following the guide, feel free to provide additional context or details. An Ultralytics engineer will be along shortly to assist you further.

If this is a 🐛 Bug Report related to the documentation, please provide a minimum reproducible example, as it greatly aids in debugging the problem.

For real-time chat and quick questions, join us on Discord 🎧. Alternatively, participate in in-depth discussions on Discourse or share your thoughts on our Subreddit.

Upgrade

Before diving into any resolution steps, ensure you're using the latest version of the ultralytics package along with its requirements in an environment with Python>=3.8 and PyTorch>=1.8:

pip install -U ultralytics

Environments

Consider running YOLO in one of the following verified environments with pre-installed dependencies:

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. Refer to our GCP Quickstart Guide
• Amazon Deep Learning AMI. See the AWS Quickstart Guide
• Docker Image. Follow our Docker Quickstart Guide

Status

Check this badge to verify if all Ultralytics CI tests are passing.

Happy coding! 🎉

[IruhDegSet]

hi,
I have an issue with my dataset which is imbalanced, I tried many techniques like data augmentaion but the problem is that
with object detection you can't really oversample one classe without affecting other classes in the same image
for example face and eyes detection, the face have two eyes, so the nuber of instances of eyes will be always as twice as the number of faces, even if you try augmentation you still get the same imbalance dataset.
how to Deal with that, if any one has an idea please share it with me??

[glenn-jocher]

@IruhDegSet balancing inherently dependent classes like faces and eyes in object detection is challenging. A potential approach is to use class weighting during training to emphasize underrepresented classes or adjust loss functions to account for class importance. For further insights, explore data preprocessing techniques. Let us know if you need more details!