How to visualize the prediction

[JunbongJang]

Hi

When I run the following command,

python train_net.py --config-file configs/Dance_R_101_3x.yaml --eval-only MODEL.WEIGHTS ./output/r101_3x_model_final.pth 

DANCE model predicts box coordinates and classes but does not produce a segmentation mask, as shown below.

How do I get the segmentation?
Also, which code did you use to visualize the predictions, like the figures in your DANCE paper?

Thank you for your help in advance.

[JunbongJang]

Dear all,

1. To get the segmentation, look for the "pred_polys" key among the output of the model in the following function in the detectron2/detectron2/evaluation/evaluator.py

inference_on_dataset(model, data_loader, evaluator)

2. There is a visualization function in the dance/core/modeling/edge_snake/snake_head.py

vis(image, poly_sample_locations, poly_sample_targets)

If you write some code with two information above, you can get the following image:

p.s. I had to put the following code in the line 98 of /detectron2/detectron2/data/dataset_mapper.py in order to correctly visualize the predicted boxes and segmentation on the image.

dataset_dict['height'] = image_shape[0]
dataset_dict['width'] = image_shape[1]

Without it, you will get the following image instead.

[lkevinzc]

Hi @JunbongJang , sorry for the delayed reply, just saw you have already found a way to obtain the vertices point, nice!

Yes all the predictions need proper scaling since the training / testing operate on some fixed scales.

If you have further questions welcome to leave an issue or email me :)

[JunbongJang]

No Problem! @lkevinzc

Thank you for your work.

Your framework will be very useful for my future project.

[bamboopu]

Dear all,

1. To get the segmentation, look for the "pred_polys" key among the output of the model in the following function in the detectron2/detectron2/evaluation/evaluator.py

inference_on_dataset(model, data_loader, evaluator)

2. There is a visualization function in the dance/core/modeling/edge_snake/snake_head.py

vis(image, poly_sample_locations, poly_sample_targets)

If you write some code with two information above, you can get the following image:

p.s. I had to put the following code in the line 98 of /detectron2/detectron2/data/dataset_mapper.py in order to correctly visualize the predicted boxes and segmentation on the image.

dataset_dict['height'] = image_shape[0]
dataset_dict['width'] = image_shape[1]

Without it, you will get the following image instead.

Thanks for your sharing. It works well.