High CPU usage, low GPU usage

[raj-kapil]

I am running my model inference over some video simultaneously, so my CPU usage goes close to 95-100% while GPU utilization is close to ~40% only. The below code is in a multi-processing function -

results = model.track(source=INPUT_VIDEO_PATH, conf=0.8, tracker="bytetrack.yaml", persist=True, stream=True, project=folder_path, device=0)

What should I do in order to utilize the GPU more? Please let me know if more info is required in answering the question, I will gladly edit and include more info.

[glenn-jocher]

@raj-kapil it sounds like you're experiencing a bottleneck that's causing high CPU usage during video inference with YOLOv8. This can happen for various reasons, such as I/O operations, data preprocessing, or other non-GPU accelerated tasks.

To address this, consider the following suggestions:

1. Batch Size: Increase the batch size if possible, as larger batches can better utilize GPU resources.

2. Data Loading: Ensure that data loading is not the bottleneck. Using a faster storage solution or optimizing the data pipeline can help.

3. Preprocessing: Check if the preprocessing steps are optimized and not causing the CPU to be the bottleneck.

4. Multi-threading: If not already implemented, consider using multi-threading for I/O operations to reduce the CPU load.

5. Hardware Check: Ensure that your system's CPU-GPU bandwidth is not a limiting factor.

6. Model Optimization: If the model itself is not fully utilizing the GPU, you might want to look into model optimization techniques.

7. System Monitoring: Use system monitoring tools to identify the exact processes that are causing high CPU usage.

8. Profiling: Profile your code to find out where the bottlenecks are occurring.

If after these checks and optimizations you still face issues, please provide more details about your system configuration, the size of the videos, and the preprocessing steps you are using. This will help in diagnosing the problem more effectively. For further guidance on optimizing performance, you can refer to the Benchmark section in our documentation.