The Movidius™ Neural Compute Stick (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. NCS is powered by the same low power high performance Movidius™ Vision Processing Unit (VPU) that can be found in millions of smart security cameras, gesture controlled drones, industrial machine vision equipment, and more.
This project is a ROS wrapper for NC API of NCSDK, providing the following features:
- A ROS service for object classification and detection of a static image file
- A ROS publisher for object classification and detection of a video stream from a RGB camera
- Demo applications to show the capabilities of ROS service and publisher
- Support multiple CNN models of Caffe and Tensorflow
There are 2 active branches in this project:
- master - stable branch
The latest version on it is v0.5.0 which supports NCSDK v1.11.00. master branch is only updated when every milestone release ready. - devel - default branch
This branch is updated from time to time and maintain the latest code on it. Each pull request should be submitted based on devel branch. We will merge patches to master branch on every milestone release.
- An x86_64 computer running Ubuntu 16.04
- ROS Kinetic
- Movidius Neural Compute Stick (NCS)
- Movidius Neural Compute (MvNC) SDK
- Movidius Neural Compute Application Zoo
- RGB Camera, e.g. RealSense D400 Series or standard USB camera
- Install ROS Kinetic Desktop-Full (guide)
- Create a catkin workspace (guide)
- Install NCSDK v1.11.00 (github)
- Install NC APP Zoo(github)
- NCSDK should be installed in
/opt/movidiusby default. Create a symbol link in/opt/movidiusto NC APP Zoo.
sudo ln -s <your-workspace>/ncappzoo /opt/movidius/ncappzooAfter that, make sure you can find graph data in /opt/movidius/ncappzoo/caffe or /opt/movidius/ncappzoo/tensorflow and image data in /opt/movidius/ncappzoo/data/images
- Install ROS package for different cameras as needed. e.g.
- Standard USB camera
sudo apt-get install ros-kinetic-usb-cam
- RealSense D400 series camera
- Install Intel® RealSense™ SDK 2.0 (guide). Refer here for more details about Intel® RealSense™ SDK 2.0.
- Install Intel® RealSense™ ROS (guide)
cd ~/catkin_ws/src git clone https://github.com/intel-ros/realsense.git cd realsense git checkout 2.0.0 cd ~/catkin_ws catkin_make
# Building
cd ~/catkin_ws/src
git clone https://github.com/intel/ros_intel_movidius_ncs.git
cd ros_intel_movidius_ncs
git checkout master
cd ~/catkin_ws
catkin_make
# Installation
catkin_make install
source install/setup.bash
# Copy label files from this project to the installation location of NCSDK
cp ~/catkin_ws/src/ros_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/| CNN Model | Framework | Usage |
|---|---|---|
| AlexNet | Caffe | Image/Video |
| GoogleNet | Caffe | Image/Video |
| SqueezeNet | Caffe | Image/Video |
| Inception_v1 | Tensorflow | Image/Video |
| Inception_v2 | Tensorflow | Image/Video |
| Inception_v3 | Tensorflow | Image/Video |
| Inception_v4 | Tensorflow | Image/Video |
| MobileNet | Tensorflow | Image/Video |
| CNN Model | Framework | Usage |
|---|---|---|
| MobileNetSSD | Caffe | Image/Video |
| TinyYolo_v1 | Caffe | Image/Video |
Classification: /movidius_ncs_nodelet/classified_objects
Detection: /movidius_ncs_nodelet/detected_objects
Classification: /movidius_ncs_image/classify_object
Detection: /movidius_ncs_image/detect_object
- Only absolute path of image file supported in image inference demo
- Only test on RealSense D400 series camera and Microsoft HD-300 USB camera
- Current v0.5.0 supporting NCSDK v1.11.00 is on master branch. devel branch is the development branch for the next release.
- Support multiple NCS devices
- Support more CNN models
- Support latest NCSDK