first commit

dvlab-research · Aug 3, 2020 · f4f5560 · f4f5560
commit f4f5560
Show file tree

Hide file tree

Showing 15 changed files with 2,733 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -0,0 +1,82 @@
+## PFENet
+This is the implementation of our paper **PFENet: Prior Guided Feature Enrichment Network for Few-shot Segmentation** that has been accepted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).
+
+## Get Started
+
+### Environment
++ torch==1.4.0 (torch version >= 1.0.1.post2 should be okay to run this repo)
++ numpy==1.18.4
++ tensorboardX==1.8
++ cv2==4.2.0
+
+
+### Datasets and Data Preparation
+
++ PASCAL-5i is based on the PASCAL VOC 2012 (http://host.robots.ox.ac.uk/pascal/VOC/voc2012/) and SBD (http://home.bharathh.info/pubs/codes/SBD/download.html) where the val images should be excluded.
+
++ COCO: https://cocodataset.org/#download.
+
+This code reads data from .txt files where each line contains the paths for image and the correcponding label respectively. Image and label paths are seperated by a space. Example is as follows:
+
+    image_path_1 label_path_1
+
+    image_path_2 label_path_2
+
+    image_path_3 label_path_3
+
+    ...
+
+    image_path_n label_path_n
+
+Then update the train/val/test list paths in the config files.
+
+### Run Demo / Test with Pretrained Models
++ Please download the pretrained models.
++ We provide **8 pre-trained models**: 4 ResNet-50 based [**models**](https://mycuhk-my.sharepoint.com/:u:/g/personal/1155122171_link_cuhk_edu_hk/EW20i_eiTINDgJDqUqikNR4Bo-7kVFkLBkxGZ2_uorOJcw?e=4%3aSIRlwD&at=9) for PASCAL-5i and 4 VGG-16 based [**models**](https://mycuhk-my.sharepoint.com/:u:/g/personal/1155122171_link_cuhk_edu_hk/EYS498D4TOZMtIb3WbQDGSQBsqxJHLSiMEAa49Iym0NO0A?e=4%3apRTPnj&at=9) for COCO.
++ Update the config file by speficifying the target **split** and **path** (`weights`) for loading the checkpoint.
++ Execute `mkdir initmodel` at the root directory.
++ Download the ImageNet pretrained [**backbones**](https://mycuhk-my.sharepoint.com/:u:/g/personal/1155122171_link_cuhk_edu_hk/EQEY0JxITwVHisdVzusEqNUBNsf1CT8MsALdahUhaHrhlw?e=4%3a2o3XTL&at=9) and put them into the `initmodel` directory.
++ Then execute the command: 
+
+    `sh test_PFENet.sh {*dataset*} {*model_config*}`
+
+Example: Test PFENet with ResNet50 on the split 0 of PASCAL-5i: 
+
+    sh test_PFENet.sh pascal split0_resnet50
+
+
+### Train
+
+Execute this command at the root directory: 
+
+    sh train_PFENet.sh {*dataset*} {*model_config*}
+
+
+## Related Repositories
+
+This project is built upon a very early version of **SemSeg**: https://github.com/hszhao/semseg. 
+
+Other projects in few-shot segmentation:
++ OSLSM: https://github.com/lzzcd001/OSLSM
++ CANet: https://github.com/icoz69/CaNet
++ PANet: https://github.com/kaixin96/PANet
++ FSS-1000: https://github.com/HKUSTCV/FSS-1000
++ AMP: https://github.com/MSiam/AdaptiveMaskedProxies
++ On the Texture Bias for FS Seg: https://github.com/rezazad68/fewshot-segmentation
++ SG-One: https://github.com/xiaomengyc/SG-One
++ FS Seg Propogation with Guided Networks: https://github.com/shelhamer/revolver
+
+
+Many thanks to their greak work!
+
+## Citation
+
+If you find this project useful, please consider citing:
+```
+@article{tian2020pfenet,
+  title={Prior Guided Feature Enrichment Netowkr for Few-Shot Segmentation},
+  author={Tian, Zhuotao and Zhao, Hengshuang and Shu, Michelle and Yang, Zhicheng and Li, Ruiyu and Jia, Jiaya},
+  journal={TPAMI},
+  year={2020}
+}
+```
diff --git a/config/coco/coco_split0_resnet101.yaml b/config/coco/coco_split0_resnet101.yaml
@@ -0,0 +1,62 @@
+DATA:
+  data_root: 
+  train_list: 
+  val_list: 
+  classes: 2
+
+TRAIN:
+  layers: 101 # 50 or 101
+  sync_bn: False
+  train_h: 641
+  train_w: 641
+  val_size: 641
+  scale_min: 0.8  # minimum random scale
+  scale_max: 1.25 # maximum random scale
+  rotate_min: -10  # minimum random rotate
+  rotate_max: 10  # maximum random rotate
+  zoom_factor: 8  # zoom factor for final prediction during training, be in [1, 2, 4, 8]
+  ignore_label: 255
+  padding_label: 255
+  aux_weight: 1.0
+  train_gpu: [0,1,2,3]  # If only one gpu is used, batch size can be set to 8 and base_lr should be 0.005.
+  workers: 16  # data loader workers
+  batch_size: 32  # batch size for training. 
+  batch_size_val: 1 # this version of code only support val batch = 1
+  base_lr: 0.02
+  epochs: 50
+  start_epoch: 0
+  power: 0.9 # 0 means no decay
+  momentum: 0.9
+  weight_decay: 0.0001
+  manual_seed: 321
+  print_freq: 5
+  save_freq: 20
+  save_path: exp/coco/split0_resnet101/model
+  weight:  
+  resume:  # path to latest checkpoint (default: none)
+  evaluate: True
+  split: 0
+  shot: 1 
+  vgg: False  # whether to use vgg as the backbone
+  ppm_scales: [1.0, 0.5, 0.25, 0.125]
+  fix_random_seed_val: True
+  warmup: False
+  use_coco: True
+  use_split_coco: True
+  resized_val: True
+  ori_resize: True  # use original label for evaluation
+
+## deprecated multi-processing training
+Distributed:
+  dist_url: tcp://127.0.0.1:6789
+  dist_backend: 'nccl'
+  multiprocessing_distributed: False
+  world_size: 1
+  rank: 0
+  use_apex: False
+  opt_level: 'O0'
+  keep_batchnorm_fp32:
+  loss_scale:
+
+
+
diff --git a/config/coco/coco_split0_vgg.yaml b/config/coco/coco_split0_vgg.yaml
@@ -0,0 +1,62 @@
+DATA:
+  data_root: 
+  train_list: 
+  val_list: 
+  classes: 2
+
+TRAIN:
+  layers: 101 # 50 or 101
+  sync_bn: False
+  train_h: 641
+  train_w: 641
+  val_size: 641  
+  scale_min: 0.8  # minimum random scale
+  scale_max: 1.25 # maximum random scale
+  rotate_min: -10  # minimum random rotate
+  rotate_max: 10  # maximum random rotate
+  zoom_factor: 8  # zoom factor for final prediction during training, be in [1, 2, 4, 8]
+  ignore_label: 255
+  padding_label: 255
+  aux_weight: 1.0
+  train_gpu: [0,1,2,3]  # If only one gpu is used, batch size can be set to 8 and base_lr should be 0.005.
+  workers: 16  # data loader workers
+  batch_size: 32  # batch size for training. 
+  batch_size_val: 1 # this version of code only support val batch = 1
+  base_lr: 0.02
+  epochs: 50
+  start_epoch: 0
+  power: 0.9 # 0 means no decay
+  momentum: 0.9
+  weight_decay: 0.0001
+  manual_seed: 321
+  print_freq: 5
+  save_freq: 20
+  save_path: exp/coco/split0_vgg/model
+  weight:  
+  resume:  # path to latest checkpoint (default: none)
+  evaluate: True
+  split: 0
+  shot: 1 
+  vgg: True  # whether to use vgg as the backbone
+  ppm_scales: [1.0, 0.5, 0.25, 0.125]
+  fix_random_seed_val: True
+  warmup: False
+  use_coco: True
+  use_split_coco: True
+  resized_val: True
+  ori_resize: True  # use original label for evaluation
+
+## deprecated multi-processing training
+Distributed:
+  dist_url: tcp://127.0.0.1:6789
+  dist_backend: 'nccl'
+  multiprocessing_distributed: False
+  world_size: 1
+  rank: 0
+  use_apex: False
+  opt_level: 'O0'
+  keep_batchnorm_fp32:
+  loss_scale:
+
+
+
diff --git a/config/pascal/pascal_split0_resnet50.yaml b/config/pascal/pascal_split0_resnet50.yaml
@@ -0,0 +1,61 @@
+DATA:
+  data_root: 
+  train_list: 
+  val_list: 
+  classes: 2
+
+
+TRAIN:
+  layers: 50
+  sync_bn: False
+  train_h: 473
+  train_w: 473
+  val_size: 473
+  scale_min: 0.9  # minimum random scale
+  scale_max: 1.1 # maximum random scale
+  rotate_min: -10  # minimum random rotate
+  rotate_max: 10  # maximum random rotate
+  zoom_factor: 8  # zoom factor for final prediction during training, be in [1, 2, 4, 8]
+  ignore_label: 255
+  padding_label: 255
+  aux_weight: 1.0
+  train_gpu: [0]
+  workers: 8  # data loader workers
+  batch_size: 4  # batch size for training
+  batch_size_val: 1
+  base_lr: 0.0025
+  epochs: 200
+  start_epoch: 0
+  power: 0.9 # 0 means no decay
+  momentum: 0.9
+  weight_decay: 0.0001
+  manual_seed: 321
+  print_freq: 5
+  save_freq: 20
+  save_path: exp/pascal/split0_resnet50/model
+  weight:  # load weight for fine-tuning or testing
+  resume:  # path to latest checkpoint (default: none)
+  evaluate: True
+  split: 0
+  shot: 1
+  vgg: False
+  ppm_scales: [60, 30, 15, 8]
+  fix_random_seed_val: True
+  warmup: False
+  use_coco: False
+  use_split_coco: False
+  resized_val: True
+  ori_resize: True  # use original label for evaluation
+
+## deprecated multi-processing training
+Distributed:
+  dist_url: tcp://127.0.0.1:6789
+  dist_backend: 'nccl'
+  multiprocessing_distributed: False
+  world_size: 1
+  rank: 0
+  use_apex: False
+  opt_level: 'O0'
+  keep_batchnorm_fp32:
+  loss_scale:
+