zh/models/yolo-world/

[giscus[bot]]

zh/models/yolo-world/

发现YOLO-World，这是一个基于YOLOv8 的框架，用于实时检测图像中的开放词汇对象。它增强了用户互动，提高了计算效率，并能适应各种视觉任务。

https://docs.ultralytics.com/zh/models/yolo-world/

[liuyiche]

测试的效果似乎比hugging face上的差不少，这是什么原因呢

[pderrenger]

感谢您的提问！🙂 如果您发现测试效果与Hugging Face上的模型相比有所不同，这可能是由于几个因素造成的。首先，请确保您使用的是最新版本的YOLOv8模型，并且已经正确加载了预训练权重。此外，不同的测试环境、数据预处理步骤以及评估指标的选择也可能导致性能差异。

YOLO-World模型特别设计用于实时开放词汇检测任务，它通过视觉语言建模和大量数据集上的预训练，实现了高效的物体识别。如果您在特定任务上遇到性能问题，尝试调整输入图像的大小或使用不同的模型变体（如YOLOv8s-world、YOLOv8m-world或YOLOv8l-world）可能会有所帮助。

此外，您可以利用YOLO-World的“先提示后检测”策略，通过自定义提示来优化模型的检测效果。例如：

from ultralytics import YOLO

# 初始化YOLO-World模型
model = YOLO('yolov8s-world.pt')

# 定义自定义类别
model.set_classes(["person", "bus"])

# 在指定图像上执行预测
results = model.predict('path/to/image.jpg')

# 展示结果
results[0].show()

希望这些建议能帮助您改善模型的性能！如果您有任何进一步的问题或需要更多帮助，请随时告诉我们。

[chnyhz]

I think it's because the ultralytics model was only trained on the coco8 dataset.

[glenn-jocher]

@chnyhz the performance difference might be due to the dataset used for training. The YOLO-World model was trained on the COCO8 dataset, which is smaller and may not cover as many variations as other datasets. For improved results, consider using a more extensive dataset or fine-tuning the model on your specific data.

[hhuiwang]

model = YOLOWorld("yolov8s-worldv2.yaml")
你好，这个yolov8s-worldv2.yaml文件在哪？我怎么制作自定义的yaml进行训练。最小的预训练型号为S,但是在我的使用场景中，S还是太大了。我该怎么降低模型参数？我希望可以手动调节 [depth, width, max_channels] 这些参数。

[pderrenger]

你好！👋

对于想要自定义 yolov8s-worldv2.yaml 文件和手动调整模型参数的需求，可以按照以下简洁的步骤进行：

1. yolov8s-worldv2.yaml 文件通常位于我们的模型配置目录中，你可以在这里找到最新的预训练模型配置：Ultralytics GitHub。

2. 要制作自己的 .yaml 文件并调整模型参数（如 depth、width、max_channels），你只需复制一个现有模型的 .yaml 文件，然后在其中根据需求修改相应配置即可。例如：

   # 根据 yolov8s-worldv2.yaml 创建 custom-model.yaml 并开始调整
   depth_multiple: 0.33  # 模型深度 (默认)
   width_multiple: 0.50  # 椎体宽度 (默认)

3. 对 .yaml 中的 depth_multiple 和 width_multiple 进行修改可以减小模型的大小。将这些值设定为低于默认值（如例子中的值）将减少模型的深度和宽度，从而减小模型参数的数量。

4. 如果你觉得 S 型号对你的使用场景而言还是太大，通过调整上述参数你可以进一步减小模型的规模。

5. 完成上述步骤后，保存 .yaml 文件，并在训练时指定它来使用自定义的配置。

希望这可以帮到你！如果有更多问题，欢迎随时提问。

[jude-zengqingxi]

@hhuiwang 请问有找到这个配置文件吗，我这边也是碰到了一样的情况

[goodbbboy]

Hello, does the current code support training a new open set detection model using my own datasets like YOLO-World? Or just use the network structure of YOLO-World for closed settraining? thakns！！

[glenn-jocher]

Hey there! 👋 YOLOv8 and YOLO-World currently focus primarily on detection tasks within the available pre-trained models, such as yolov8s-world.pt for object detection. These models are optimized for high performance and efficiency in open-vocabulary object detection tasks.

For training new models using your own datasets, the current setup, particularly with the YOLO-World models (yolov8s-worldv2.pt, etc.), is more skewed towards using these pretrained models for inference. The training on custom datasets specifically for open set detection, akin to what YOLO-World demonstrates, isn't directly supported in the out-of-the-box YOLOv8 implementation.

However, you can certainly use the YOLOv8 architecture as a base for closed-set training on your datasets. You could adapt YOLOv8 models for your custom training needs by preparing your dataset similar to expected formats and tuning the training pipeline accordingly.

An example of training a YOLOv8 model on a custom dataset would look something like this in Python:

from ultralytics import YOLO

# Load YOLOv8 model configuration and pretrained weights
model = YOLO('yolov8n.yaml').load('yolov8n.pt')

# Train the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)

And for CLI, resembling:

yolo train model=yolov8n.yaml data=your_custom_dataset.yaml epochs=100 imgsz=640

Please ensure your dataset is properly formatted according to YOLOv8 expectations (similar to how COCO or VOC datasets are structured).

Always happy to help if you have more questions or need further guidance! 🌟

[goodbbboy]

Thank you for your reply. Looking forward to your more interesting work！！

[glenn-jocher]

Hey there! Thanks a lot for your excitement and kind words! 🌟

Regarding your question, yes, you can certainly leverage the YOLO-World structure for closed-set training using your custom datasets. Although the YOLO-World models excel in open-vocabulary object detection, you're absolutely free to use these models as a powerful backbone for your specific detection tasks.

For training on your datasets, you'd typically follow the standard custom training approach of YOLOv8 with your dataset formatted accordingly. Here's a quick example to get you started:

from ultralytics import YOLO

# Initialize the model
model = YOLO('yolov8s-worldv2.pt')  # Using the YOLO-World pre-trained model

# Train on your custom dataset
model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)

Just remember to thoroughly prepare your dataset in a compatible format. If you’re diving into a more specific open-vocabulary direction or wish to explore more possibilities, tweaking and experimenting with the model and dataset would be key. 😊

Excited to see what amazing things you'll build! If you have any more questions or run into any roadblocks, feel free to reach out. Happy coding!

[00mmw]

How can I use yolo-world to segment and detect the instances?

[glenn-jocher]

To segment and detect instances using YOLO-World, you can leverage the power of YOLOv8's object detection capabilities enhanced for open-vocabulary tasks. Although direct "segmentation" in the traditional sense (like per-pixel classification) isn't explicitly mentioned for YOLO-World, YOLOv8 models are inherently capable of detecting and partially segmenting objects through bounding boxes. Here's a quick example of using YOLO-World for object detection, which can be similarly applied for segmenting instances by bounding areas:

from ultralytics import YOLO

# Load a pretrained YOLO-World model
model = YOLO('yolov8s-worldv2.pt')  # Choose the appropriate model

# Run detection. The model can directly recognize and predict open-vocabulary objects.
results = model.predict('path/to/your/image.jpg')

# Display detected objects and their bounding boxes
results[0].show()

For tasks requiring more detailed segmentation (like instance or semantic segmentation), you might need to adapt or complement YOLO-World with additional segmentation models or techniques depending on your specific requirements. The open-vocabulary detection capability of YOLO-World, as described, significantly enhances its utility in dynamic scenes and for detecting a wide range of objects described by descriptive texts.

Keep in mind, YOLO-World excels in the area of detection with its efficient open-vocabulary capabilities, providing a unique tool for various vision-based applications.

[hreeng-cyber]

您好 我想自己定义一个新的类别做零样本检测，
from ultralytics import YOLO

加载预训练模型

model = YOLO('yolov8s-world.pt')

定义检测类别

model.set_classes(["defect"])

保存模型

model.save("custom_yolov8s.pt")
但是用新的模型检测我的数据集一个缺陷都检测不到，请问应该怎么调整呢

[glenn-jocher]

@hreeng-cyber to improve detection results with your custom class, ensure your dataset is well-labeled and diverse. You might also need to fine-tune the model on your specific dataset to enhance performance. If issues persist, consider adjusting hyperparameters or using a larger model variant.

[hreeng-cyber]

但是如果做零样本推理的话，上述的代码逻辑是正确的吗？我想先看一下零样本推理的效果再做微调，但是通过上述代码保存的模型去检测“defect”新类别，每张图片都没有结果。是代码的问题吗？
【from ultralytics import YOLO
from PIL import Image
import os

model = YOLO('custom_yolov8s.pt')

images_folder = 'data/shanghai/images/dataset/1'
results_folder = 'data/shanghai/result/dataset/1'

if not os.path.exists(results_folder):
os.makedirs(results_folder)

for filename in os.listdir(images_folder):
if filename.lower().endswith(('.png', '.jpg', '.jpeg')):
img_path = os.path.join(images_folder, filename)
results = model.predict(img_path)

    result_img_path = os.path.join(results_folder, filename)  
    results[0].save(result_img_path)  】

[glenn-jocher]

To improve zero-shot detection results, ensure your custom class "defect" is well-represented in the offline vocabulary. You might need to adjust the model's prompts or fine-tune it with relevant data to enhance detection accuracy.

[YeahTOT]

Hello, is it possible to use YOLOWorld to detect unknown classes that are not present in the dataset (for example, if the dataset contains only the 'car' class, but I want to detect both 'cars' and all other classes)? How can I fine-tune it on my own dataset for this purpose?

[YeahTOT]

For example, I want to add an "others" class.

[glenn-jocher]

@YeahTOT yOLO-World can detect unknown classes using its open-vocabulary capabilities. You can set custom prompts to include an "others" class without retraining. For fine-tuning on your dataset, use the train method with your data. More details are available in the YOLO-World documentation.

[98bubu]

from ultralytics import YOLOWorld
from ultralytics.models.yolo.world.train_world import WorldTrainerFromScratch

data = {
"train": {
"yolo_data": ["coco.yaml"],
"grounding_data": [
{
"img_path": "../datasets/flickr30k/images",
"json_file": "../datasets/flickr30k/final_flickr_separateGT_train.json",
},

    ],
},
"val": {"yolo_data": ["coco.yaml"]},

}

model = YOLOWorld("yolov8s-worldv2.yaml")
model.train(data=data, batch=128, epochs=100, trainer=WorldTrainerFromScratch)
你好，我想问一下我这样训练之后为什么MAP50的精度只有0.045左右？这是什么问题呢

[98bubu]

您好，我想请问一下是否有YOLOV8n-worldv2在COCO数据集上面的训练结果？期待您的答复。

[98bubu]

此外还想请问一下yolo-world文档里面提供的在coco上面的结果是用普通train方法仅训练了coco数据集，还是使用WorldTrainerFromScratch方法加入了grounding_data？如果加入了是用的上面提供的flickr30k数据集吗？期待得到您的答复，不胜感激！

[glenn-jocher]

在COCO数据集上的YOLO-World结果是通过普通train方法仅使用COCO数据集进行训练的，并未加入grounding_data。如果您希望加入grounding_data（如Flickr30k），需要使用WorldTrainerFromScratch方法，这会提升模型的多模态能力和开放词汇检测性能。更多细节可参考YOLO-World文档。

[98bubu]

我在YOLO-World文档里面没有看见YOLOV8n-worldv2在COCO数据集上面的训练结果，请问一下您在YOLOV8n-worldv2上的效果如何呢？期待得到您的答复，不胜感激！

[glenn-jocher]

根据官方文档，目前未提供 YOLOv8n-worldv2 在 COCO 数据集上的具体训练结果，但相较于其他 YOLO-World 模型，其结果可能会略低。您可以参考文档中的其他模型性能，并尝试复现相关实验。

[hm123450]

您好，我想问一下，为什么我本地运行的yolov8x-worldv2检测不出来door，但是demo里面能检测出来？

[glenn-jocher]

您好，yolov8x-worldv2无法检测到“door”可能与模型的离线词汇或提示设置有关。建议您尝试通过model.set_classes(["door"])添加自定义检测类别，或者检查输入的描述性文本是否与demo中使用的一致。更多详情可参考YOLO-World文档。

[andwizard]

之后是否会有微调相关的教程呢

[glenn-jocher]

@andwizard 感谢您的关注！关于微调相关的教程，目前您可以参考 Ultralytics 文档 中的训练部分，了解如何使用 train 方法进行模型微调。如果有进一步的更新或教程发布，我们会及时在文档或此仓库中公布。

[9527-csroad]

Hello, I use the YOLOWorld to predict open word, but it not work and it detect nothing. Could you help me to found the problems? thank you

from ultralytics import YOLOWorld
model = YOLOWorld('/noya/coco/models/yolov8s-world.pt')
model.set_classes(["nose", "hair", "lip", "beautiful", "eyebrow"])
results = model.predict('/noya/coco/test_images/1.jpg')

image 1/1 /noya/coco/test_images/2.png: 640x640 (no detections), 8.2ms
Speed: 2.3ms preprocess, 8.2ms inference, 0.7ms postprocess per image at shape (1, 3, 640, 640)

[glenn-jocher]

Thanks for reaching out! Let's troubleshoot your YOLO-World detection issue. First, ensure your custom classes like "nose" and "eyebrow" are visible/clear in your test image. Try adjusting the confidence threshold with model.predict(source, conf=0.3) as some detections might be filtered out. For abstract concepts like "beautiful", consider using more concrete object descriptors.

If issues persist, we recommend using the newer yolov8s-worldv2.pt model which shows improved performance (see COCO benchmark table in our docs). Let us know if this helps! 🚀