Skip to content

Latest commit

 

History

History
195 lines (166 loc) · 7.47 KB

README.md

File metadata and controls

195 lines (166 loc) · 7.47 KB
Raw

SphereVLAD Series: Viewpoint-invariant Place Recognition

1. Pytorch implementation SphereVLAD and SphereVLAD++ proposed in the papers:
SeqSphereVLAD: Sequence Matching Enhanced Orientation-invariant Place Recognition
Peng Yin, Fuying Wang, Anton Egorov, Jiafan Hou, Ji Zhang, Howie Choset
SphereVLAD++: Attention-based and Signal-enhanced Viewpoint Invariant Descriptor
Shiqi Zhao, Peng Yin*, Ge Yi, Sebastian Scherer

2. Core library for our iSimLoc, AutoMerge and BioSLAM works:
iSimLoc: Visual Global Localization for Previously Unseen Environments With Simulated Images
Peng Yin; Ivan Cisneros; Shiqi Zhao; Ji Zhang; Howie Choset; Sebastian Scherer
AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale Environments
Peng Yin; Shiqi Zhao; Haowen Lai; Ruohai Ge; Ji Zhang; Howie Choset; Sebastian Scherer
BioSLAM: A Bioinspired Lifelong Memory System for General Place Recognition
Peng Yin; Abulikemu Abuduweili; Shiqi Zhao; Lingyun Xu; Changliu Liu; Sebastian Scherer

Network Structure

Environment Dependency

System

The codes are tested in:

  • Ubuntu20.04+CUDA11.6+python3.8.10
  • Ubuntu18.04+CUDA10.2+python3.6.9

Python Environment Dependency

  • s2cnn
  • torch==1.13.0+cu116 torchaudio==0.13.0+cu116 torchvision==0.14+cu116
  • cupy-cuda11x==11.6.0

Installation

It is highly recommended too create a virtual environment by conda or virtualenv.

# create virtual env
virtualenv /path_to_env/spherevlad -p /usr/bin/python3
source /path_to_env/spherevlad/bin/activate
# or
conda create -n spherevlad python==3.8
conda activate spherevlad

# install pytorch and cupy
pip3 install torch==1.13.0+cu116 torchvision==0.14.0+cu116 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu116
pip3 install cupy-cuda11x==11.6.0

# install s2cnn
pip3 install pynvrtc joblib
mkdir third && cd third
git clone https://github.com/jonkhler/s2cnn.git
cd s2cnn
python setup.py install

# install all others packages
pip3 install -r requirements.txt

Lidar Place Recognition

Data Structure

├── data 
    ├── results
    │   └── model_name
    │       ├── DATA.yaml
    │       ├── MODEL.yaml
    │       ├── log.txt
    │       └── pth
    │           ├── model_xxx.pth
    │           ├── ...
    └── dataset
        ├── PITT
        │   ├── DATA
        │   │   ├── Train1
        │   │   │   ├── cloudGlobal.pcd
        │   │   │   └── poses.txt
        │   │   ├── ...
        │   │   └── Trainxx

Note

Modify the PYTHONPATH environment variable to include absolute path to the project root folder:

export PYTHONPATH=$PATHONPATH:/path_to_spherevlad/SphereVLAD

Data Preparation

Download the sample data from Dropbox and put unzipped folder in the /data/PITT. The data structure should be like the same mentioned in Data Structure section.

mkdir data
python generating_sph/gene_pitt.py --config-network=config/network/spherevlad.yaml --config-dataset=config/dataset/PITT.yaml

Training on Alita Urban Dataset

# spherevlad
python3 train_lcd.py --config-network=config/network/spherevlad.yaml --config-dataset=config/dataset/PITT.yaml
# spherevlad2
python3 train_lcd.py --config-network=config/network/spherevlad2.yaml --config-dataset=config/dataset/PITT.yaml

  • If there are issues with the s2cnn, please download the version we have modified and tested.

  • The generated weight will be saved at /data/results/SphereVLAD-{DatasetName}-{DD}-{MM}-{YYYY}-{TIME}, the folder also contains the log and configuration of the model and dataset.

Testing on Alita Urban Dataset

Pretrained weight on Alita Urban Dataset: Dropbox. Please unzip it and put in /data/results

Please change the trajectory numbers in line 45 and line 68 in eval/evaluate_pitt.py first.

# spherevlad
python eval/evaluate_pitt.py --model-path=data/results/SphereVLAD --epoch=666 --noise=1 --type=recall --trans-noise=2 --rot-noise=180 --log=False
# spherevlad2
python eval/evaluate_pitt.py --model-path=data/results/SphereVLAD2 --epoch=666 --noise=1 --type=recall --trans-noise=2 --rot-noise=180 --log=False
  • --model-path: the folder contains the configuration and trained weights
  • --epoch: the epoch of trained weights

Hints on training and test on other dataset

  • Modify the dataloader/pittsburgh.py to fit the data structure of your own dataset
  • Modify the config/dataset/PITT.yaml to change the configuration of the dataset
  • Add new evaluation methods in eval/eval_utils.py and import it in train_lcd.py

Citation

@INPROCEEDINGS{9341727,
  author={Yin, Peng and Wang, Fuying and Egorov, Anton and Hou, Jiafan and Zhang, Ji and Choset, Howie},
  booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={SeqSphereVLAD: Sequence Matching Enhanced Orientation-invariant Place Recognition}, 
  year={2020},
  volume={},
  number={},
  pages={5024-5029},
  doi={10.1109/IROS45743.2020.9341727}}

@ARTICLE{9956017,
  author={Zhao, Shiqi and Yin, Peng and Yi, Ge and Scherer, Sebastian},
  journal={IEEE Robotics and Automation Letters}, 
  title={SphereVLAD++: Attention-Based and Signal-Enhanced Viewpoint Invariant Descriptor}, 
  year={2023},
  volume={8},
  number={1},
  pages={256-263},
  doi={10.1109/LRA.2022.3223555}}

@article{yin2022alita,
  title={Alita: A large-scale incremental dataset for long-term autonomy},
  author={Yin, Peng and Zhao, Shiqi and Ge, Ruohai and Cisneros, Ivan and Fu, Ruijie and Zhang, Ji and Choset, Howie and Scherer, Sebastian},
  journal={arXiv preprint arXiv:2205.10737},
  year={2022}
}

@ARTICLE{10035430,
  author={Yin, Peng and Cisneros, Ivan and Zhao, Shiqi and Zhang, Ji and Choset, Howie and Scherer, Sebastian},
  journal={IEEE Transactions on Robotics}, 
  title={iSimLoc: Visual Global Localization for Previously Unseen Environments With Simulated Images}, 
  year={2023},
  volume={39},
  number={3},
  pages={1893-1909},
  doi={10.1109/TRO.2023.3238201}}


@ARTICLE{10203034,
  author={Yin, Peng and Zhao, Shiqi and Lai, Haowen and Ge, Ruohai and Zhang, Ji and Choset, Howie and Scherer, Sebastian},
  journal={IEEE Transactions on Robotics}, 
  title={AutoMerge: A Framework for Map Assembling and Smoothing in City-Scale Environments}, 
  year={2023},
  volume={},
  number={},
  pages={1-19},
  doi={10.1109/TRO.2023.3290448}}

@ARTICLE{10242249,
  author={Yin, Peng and Abuduweili, Abulikemu and Zhao, Shiqi and Xu, Lingyun and Liu, Changliu and Scherer, Sebastian},
  journal={IEEE Transactions on Robotics}, 
  title={BioSLAM: A Bioinspired Lifelong Memory System for General Place Recognition}, 
  year={2023},
  volume={},
  number={},
  pages={1-20},
  doi={10.1109/TRO.2023.3306615}}