During aggressive maneuvers, vision-based perception techniques tend to fail because of the lack of tracked features. The purpose of this work is to mitigate loss of feature tracks by being more clever with which features are selected for use in the optimization. By doing so, we add robustness to the visual-inertial motion estimation because features that are likely to be soon lost based on future motion are not selected. Further, we allow for more efficient optimization by using fewer features with high information content.
- L. Carlone and S. Karaman, Attention and Anticipation in Fast Visual-Inertial Navigation, 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, pp. 3886-3893. pdf
- T. Qin, P. Li and S. Shen, VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator, in IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004-1020, Aug. 2018. pdf
To see Anticipated VINS-Mono in action, make sure to download some of the EuRoC datasets.
After cloning this repo (on the anticipation branch) into your catkin workspace and building, run the following commands (change the sequence/bag name appropriately) in three separate terminals:
# remember to source!
$ roslaunch vins_estimator euroc.launch sequence_name:=MH_05_difficult# be sure to cd into the appropriate directory
rosbag play MH_05_difficult.bag --clock --pause# remember to source!
$ roslaunch vins_estimator vins_rviz.launchAnticipated VINS-Mono builds off of the publicly available VINS-Mono ROS implementation. We have added anticipation and attention components as illustrated in the block diagram below. Only the Ceres Solver-based VIO Backend block is from VINS-Mono.
The following are new files, implemented as part of the anticipation algorithm.
| File/Component | Description |
|---|---|
| feature_tracker | This is our own re-implementation of the VINS-Mono front end using the same processing ideas. Processing time is equivalent, if not 1-2 ms faster. The feature_img (PointCloud) message has been altered to return feature scores from cv::goodFeaturesToTrack. |
| state_defs.h | Useful typedefs for horizon, state, features, and information matrices. |
| feature_selector.cpp | Implements the expected information calculation (anticipation) and the feature selection algorithm (attention). This is where most of the work is done. |
| feature_selector.h | Associated header file. |
| horizon_generator.cpp | Implements the State Horizon Generator block. Two methods are implemented for predicting the future horizon: IMU, which propagates forward using the latest IMU measurement; and GT, which composes the current state estimate with relative ground truth transformations over the horizon. This method is meant to emulate having desired future poses from a path planner or from MPC. |
| horizon_generator.h | Associated header file. |
- Presentation: Google Slides
- Report: PDF