precision and recall value for different threshold(confidence) for each class

[muttahirul]

Hello good people, I trained my custom dataset on yolov8 and the saved output result accuracy metric for each class are like this:

Class Images Instances Box(P R mAP50 m
all 610 4440 0.588 0.428 0.467 0.299
auto rickshaw 610 91 0.786 0.363 0.42 0.26
bicycle 610 91 0.463 0.209 0.245 0.113
bus 610 618 0.688 0.558 0.606 0.393
car 610 890 0.658 0.634 0.665 0.449

But I want to plot precision-recall curve, precision-confidence curve and recall-confidence curve. For this I need precision and recall value for each class at different threshold(or, confidence).
how do I get precision and recall value for different threshold for each class in yolov8?

[glenn-jocher]

@muttahirul P, R, F1 and PR curves are produced automatically on final validation after training or when running val directly. These plots are saved to the your runs directory indicated at the end of training/val.

[muttahirul]

@glenn-jocher I want to edit those plot for example I want use diamond or dot marker instead of color marker. For that reason, I need the value of precision and recall @ different confidence. Are these value saved or Do I need to edit the plotting code?

[dkobayas-cyber]

@glenn-jocher I also want more flexibility in the output plot. Although the curves are produced automatically, they are not necessarily practical for the use in reports and presentations.

[glenn-jocher]

@dkobayas-cyber, the precision and recall values at different confidence thresholds are indeed calculated during the validation process, but they are not directly saved to a file by default. To customize the plots with different markers or for more flexibility in the output, you would need to modify the plotting code within the repository.

You can find the code responsible for generating the precision-recall curves in the validation section of the codebase. By adjusting this code, you can change the plot styles, markers, and other aesthetics to suit your needs for reports and presentations. If you're comfortable with Python and matplotlib (or another plotting library), you can extract the precision and recall values and create your custom plots.

Remember to adhere to the AGPL-3.0 License when modifying and using the code from the Ultralytics YOLOv8 repository. If you require further assistance or have specific questions about modifying the code, feel free to open a new issue in the repo, and we'll be glad to help!

[muttahirul]

@glenn-jocher How could I plots the P,R,F1 and PR curves using shape marker instead of color marker? As I need to print these plot in a black and white paper, color marker will not be distinguishable.

[glenn-jocher]

@muttahirul hello! To plot precision (P), recall (R), F1, and precision-recall (PR) curves using shape markers instead of color markers, you can modify the plotting function in your code to use different marker styles for each class or metric. While I can't provide code examples, I can guide you on what to consider:

1. Select Shape Markers: Choose a set of distinct shape markers (e.g., 'o', 's', 'D', '^', 'v', '<', '>', 'p', '*', 'h', 'H', '+', 'x') that will be distinguishable in black and white.

2. Modify Plotting Function: In the function responsible for plotting, you'll need to iterate over your classes or metrics and assign each one a unique shape marker.

3. Legend: Ensure your legend clearly indicates which shape corresponds to which class or metric, so the reader can easily interpret the black and white plot.

4. Consistency: Use the same shape markers consistently across all your plots for coherence.

5. Line Styles: If you're plotting curves, consider using different line styles (e.g., solid, dashed, dotted, dash-dot) in addition to shape markers for further distinction.

For more detailed guidance on the capabilities of YOLOv8, including how to work with metrics and plots, please refer to the documentation at https://docs.ultralytics.com. The documentation provides comprehensive information that can help you understand how to customize your plots according to your needs.