[English] | [中文]
CDR(cv-detect-robot)Project Introduction🔥🔥🔥（Jetson side-end optimized deployment of industrial-grade vision algorithms）
CDR Project purpose：High-performance visual inspection and its related algorithms empower the robotics industry and build a bridge for technology implementation。
- For x_86 and nvidia-jetson platforms, this project uses tensorRT optimization and builds visual detection algorithms such as yolov5, yolox and resnet based on the deepstream video streaming AI inference framework, and configures deepsort, DCF and IOU trackers for each algorithm. The detection and tracking data are transmitted by means of shared memory. Real-time data can be read in three ways: rostopic, python script and c++ program. This project provides zero-based introductory configuration and debugging tutorials, as well as docker images based on the x_86 platform and the nvidia-jetson platform, which can save the trouble of configuring the environment and import and run with one click. The image contains a full set of operating environments for the source code of the yolov5 project and a full set of operating environments for the source code of the yolox project. deepstream full set of operating environment and ros operating system (melodic)
（ Note 1）:This project has configured a full set of docker images for the ubuntu-x_86 platform and the nvidia-jetson platform. There is no need to configure a complicated software environment. One-click import and operation saves time and effort.。
（ Note 2）:In the future, this project will use Tencent Youtu's ncnn acceleration library to optimize and deploy domestic AI chips such as Huawei HiSilicon 3559, Rockchip 3399pro, Kanzhi K210 and other domestic AI chips. Please look forward to it.！！！。
（ Note 3）:With the learning and growth of myself and the team, the project will be maintained and updated from time to time. Due to the limited ability, there are errors and deficiencies in the project. Please criticize and correct or leave a message in
issue。
（ Note 4）:In order to facilitate everyone to learn and communicate, a CDR (cv-detect-robot) project communication WeChat group has been established. Please add the group leader
Xiao GuoWeChat account17370042325to facilitate pulling you into the group。
- The deepstream framework in CDR is upgraded to version 6.0 (tensorRT 8.0.1)
- All sub-projects are adapted to the deepsort tracker (IOU, DCF and deepsort three trackers can be switched at will), and the target tracking IDs of all sub-projects can be obtained through rostopic, python scripts and cpp programs。
- The whole system comes standard with docker mirroring. The project author has configured a full set of mirroring environments based on the x_86 platform and the nvidia-jetson platform for the CDR project, and comes with project mirroring operations and sub-project testing tutorials。
- The CDR image has pt->wts->trt function (yolov5) and onnx-trt function (yolox)。
- Each sub-project in the CDR image can have multi-stream detection. For details, please refer to the following link。
- Each sub-project in the CDR image has the functions of video file detection, usb camera detection and rtsp real-time stream detection.。
- The CDR image has been configured with the complete operating environment of the yolov5 project source code, the complete operating environment of the yolox project source code, the complete operating environment of the deepstream and the complete operating environment of ros-melodic. For the specific environment configuration, please refer to the following link：
CDR x86 dockerimage andCDR jetson dockerimage environment configuration and algorithm debugging are extremely complex and time-consuming, but in the spirit of open source sharing, the project author decided to make it available to the whole industry for free use：-
CDR x86 docker image(About 28G)The download link is as follows (unlimited speed)：
- Link:
http://112.74.111.51:1212/down/Dcyn8UvJ81LgExtraction code:Z78Din
- Link:
-
CDR x86 docker image dockerhub download(14.54 GB)Order：
docker pull gjy123456/cv-detect-robot:CDR-x86-v4.02
-
CDR jetson docker image(About 10G) The download link is as follows (unlimited speed)：
- Link:
http://112.74.111.51:1212/down/tSpLJEbUHvQCExtraction code:nKTzyp
- Link:
-
CDR jetson docker image(about 4.58G)dockerhubdownload command：
docker pull gjy123456/cv-detect-robot:CDR-jetson-v4.18
-
! ! ! If you use docker image, the following content in this line is only for understanding. If you want to configure the environment from scratch, you can refer to the following content! ！！！
- Introduction to the yolov5-ros-deepstream subproject
This project combines the yolov5 visual detection algorithm with the neural network acceleration engine tensorRT, and runs under the deepstream framework of NVIDIA. Combined with the ros1 communication mechanism, the detection data is released in real time through the ros topic. For developers and related researchers based on ROS robot operating system。
Please enter the detailed tutorialyolov5-ros-deepstream
Please enter the final video detection effectyolov5-ros-deepstream detect
Add the target tracker video detection effect, please enterJetson NX yolov5-ros-deepstream+Target Tracking
Add the target tracker video detection effect, please enterJetson Nano yolov5-ros-deepstream+Target Tracking
- Introduction to the yolov5-deepstream-python subproject
This project combines the visual detection algorithm yolov5 with the neural network acceleration engine tensorRT, and uses the shared memory technology to store the data obtained from the detection in the pre-defined physical memory in real time (the physical address is unique). In any software directory, create a
client.pyscript file for reading physical memory (it only contains a code segment for reading memory), and read the data in the specified physical memory. Under the premise, the code segment can be inserted into any python project that needs target detection data, so that the python project can successfully obtain the target detection data。
Please enter the detailed tutorialyolov5-deepstream-python
Please enter the final video detection effectyolov5-deepstream-python
Add the target tracker video detection effect, please enterJetson NX yolov5-python-deepstream+target tracker
Add the target tracker video detection effect, please enterJetson Nano yolov5-python-deepstream+target tracker
- Introduction to the yolov5-deepstream-cpp subproject
This project combines the visual detection algorithm yolov5 with the neural network acceleration engine tensorRT, and uses the shared memory technology to store the data obtained from the detection in the pre-defined physical memory in real time (the physical address is unique). In any software directory, create a
yolov5_tensor.cppfile for reading physical memory (it only contains a code segment for reading memory). After compiling, the data in the specified physical memory can be read out. Under the premise of success, the code segment can be inserted into any C++ project that needs target detection data, so that the C++ project can successfully obtain target detection data.
Please enter the detailed tutorialyolov5-deepstream-cpp
Please enter the final video detection effectyolov5-deepstream-cpp-detection
Add the target tracker video detection effect, please enterJetson NX yolov5-cpp-deepstream+target tracker
Add the target tracker video detection effect, please enterJetson Nano yolov5-cpp-deepstream+target tracker
- Introduction to the yolox-ros-deepstream subproject
This project combines the yolox visual detection algorithm with the neural network acceleration engine tensorRT. This sub-project adopts another engine file generation method, which is transferred to the engine through onnx. This method is more flexible and more stable, which is in line with the mainstream development of the industry. Trend, runs under NVIDIA's deepstream framework, combined with ros communication mechanism, and publishes detection data in real time through ros topic. For developers and related researchers based on ROS robot operating system.
Please enter the detailed tutorialyolox-ros-deepstream
Please enter the final video detection effectyolox-ros-deepstream-detection
Add the target tracker video detection effect, please enter[Jetson NX yolox-ros-deepstream+target tracker]](https://www.bilibili.com/video/BV1W34y1B7YB?spm_id_from=333.999.0.0)
Add the target tracker video detection effect, please enter[Jetson Nano yolox-ros-deepstream+target tracker]](https://www.bilibili.com/video/BV1Tq4y1A7km?spm_id_from=333.999.0.0)
This project uses the python interface of
deepstream, based on theresnet10.caffemodelmodel optimized by NVIDIA, and can use NVIDIA's new(TAO) Toolkittoolkit for custom data set training and model optimization, and inherits CDR The ros interface of the project heirloom. It can not only be used on Jetson series platforms, but also can be quickly deployed on Linux-x_86 platform (Ubuntu) through docker container technology. After testing, the model can achieve a frame rate of up to 30fps (detect four types of targets) on Jetson Nano when the detection algorithm and multi-target tracking algorithm are simultaneously supported, and the accuracy rate can reach more than 90% (close to yolov5)。
Please enter the detailed tutorialresnet10-ros-deepstream
Please enter the final video detection effectresnet10-ros-deepstream-detection
Add the target tracker video detection effect, please enterJetson NX resnet10-ros-deepstream+target tracker
Add the target tracker video detection effect, please enterJetson Nano resnet10-ros-deepstream+target tracker
- Introduction to the yolox-deepstream-python subproject
This project combines the yolox visual detection algorithm with the neural network acceleration engine tensorRT. This sub-project adopts another engine file generation method, which is transferred to the engine through onnx. This method is more flexible and more stable, which is in line with the mainstream development of the industry. Trend, run under NVIDIA's deepstream framework, in any software directory on the same hardware platform, create a
client.pyscript file that reads physical memory (it only contains a code segment for reading memory), will specify The data in the good physical memory is read out. On the premise of successful reading, the code segment can be inserted into any python project that needs target detection data, so that the python project can successfully obtain the target detection data.
Please enter the detailed tutorialyolox-deepstream-python
Please enter the final video detection effectyolox-deepstream-python-detection
Add the target tracker video detection effect, please enterJetson NX yolox-ros-deepstream+target tracker器
Add the target tracker video detection effect, please enterJetson Nano yolox-ros-deepstream+target tracker
- Introduction to the yolox-deepstream-cpp subproject
This project combines the yolox visual detection algorithm with the neural network acceleration engine tensorRT. This sub-project adopts another engine file generation method, which is transferred to the engine through onnx. This method is more flexible and more stable, which is in line with the mainstream development of the industry. Trend, run under NVIDIA's deepstream framework, in any software directory on the same hardware platform, create a `yolox_tensor. Read the data in the specified physical memory. On the premise of successful reading, the code segment can be inserted into any C++ project that needs target detection data, so that the C++ project can successfully obtain the target detection data.
Please enter the detailed tutorialyolox-deepstream-cpp
Please enter the final video detection effectyolox-deepstream-cpp-detection
Add the target tracker video detection effect, please enterJetson NX yolox-ros-deepstream+target tracker
Add the target tracker video detection effect, please enterJetson Nano yolox-ros-deepstream+target tracker
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker