add image detection example by louis-she · Pull Request #2520 · pytorch/ignite

louis-she · 2022-03-21T02:11:52Z

An image detection example using PyTorch Faster RCNN with Aim. Some screenshots:

vfdev-5

@louis-she thanks a lot for the PR !
This is a great start for finally adding detection example to ignite !
I left some comments on how to improve some parts of it...

examples/contrib/detection/README.md

examples/contrib/detection/faster_rcnn.py

vfdev-5 · 2022-03-21T12:33:41Z

examples/contrib/detection/faster_rcnn.py

+    @visualizer.on(Events.ITERATION_COMPLETED)
+    def add_vis_images(engine):
+        engine.state.model_outputs.append(engine.state.output)
+
+    @visualizer.on(Events.ITERATION_COMPLETED)
+    def submit_vis_images(engine):
+        aim_images = []
+        for outputs in engine.state.model_outputs:
+            for image, target, pred in zip(outputs["x"], outputs["y"], outputs["y_pred"]):
+                image = (image * 255).byte()
+                pred_labels = [Dataset.class2name[l.item()] for l in pred["labels"]]
+                pred_boxes = pred["boxes"].long()
+                image = draw_bounding_boxes(image, pred_boxes, pred_labels, colors="red")
+
+                target_labels = [Dataset.class2name[l.item()] for l in target["labels"]]
+                target_boxes = target["boxes"].long()
+                image = draw_bounding_boxes(image, target_boxes, target_labels, colors="green")
+
+                aim_images.append(aim.Image(image.numpy().transpose((1, 2, 0))))
+        aim_logger.experiment.track(aim_images, name="vis", step=trainer.state.epoch)


I'm not quite sure to understand why we need to accumulate and submit on iterations...
Can't we just submit on iterations without accumulating ?

Can submit on iterations only if we set step as trainer.state.iteration, or aim will override the previous images.

Gather visualization images in epoch can give a good view when using aim I think, but indeed it will eat more memory.

examples/contrib/detection/faster_rcnn.py

vfdev-5 · 2022-03-21T12:38:31Z

examples/contrib/detection/mean_ap.py

@@ -0,0 +1,252 @@
+"""


Just a comment for ourselves that we have to advance on implementing mAP from our side

vfdev-5 · 2022-03-21T12:39:24Z

examples/contrib/detection/mean_ap.py

+    def synchronize_between_processes(self):
+        for iou_type in self.iou_types:
+            self.eval_imgs[iou_type] = np.concatenate(self.eval_imgs[iou_type], 2)
+            create_common_coco_eval(self.coco_eval[iou_type], self.img_ids, self.eval_imgs[iou_type])


This method looks a bit weird as according to its name it should sync between procs...

sdesrozis · 2022-03-22T21:13:57Z

examples/contrib/detection/mean_ap.py

+        labels = targets["labels"].tolist()
+        areas = targets["area"].tolist()
+        iscrowd = targets["iscrowd"].tolist()
+        if "masks" in targets:


Here, the implementation could be simplified since no masks neither key points are used.

sdesrozis · 2022-03-22T21:15:41Z

examples/contrib/detection/faster_rcnn.py

+    parser.add_argument("--image-size", type=int, default=512, help="image size to train")
+    parser.add_argument("--experiment-name", type=str, default="test", help="name of one experiment")
+    args = parser.parse_args()
+    run(


idist could be used for ddp

I don't have 2 GPUs to test distribution now. I'll try to write the distribution code and, maybe test on AWS?

sdesrozis · 2022-03-22T21:20:07Z

examples/contrib/detection/faster_rcnn.py

+    name2class = {v: k + 1 for k, v in enumerate(classes)}
+    class2name = {k + 1: v for k, v in enumerate(classes)}
+
+    def __getitem__(self, index: int) -> Tuple[Any, Any]:


It seems that what is done here could be implemented as a transformation. It should be moved in another file IMO.

Do you mean the name and class number conversion? If yes I think it's ok to place here since there are nothing to do with the images.

sayantan1410 · 2022-03-25T04:41:19Z

Also, the tests are failing because of code-formatting error, try running
bash ./tests/run_code_style.sh install
bash ./tests/run_code_style.sh fmt

vfdev-5 · 2022-03-30T13:32:14Z

examples/contrib/detection/main.py

+        with torch.cuda.amp.autocast(enabled=True):
+            loss_dict = model(images, targets)
+            loss = sum(loss for loss in loss_dict.values())


Still cuda.amp.autocast, can it work on CPU ?

sdesrozis · 2022-04-03T08:47:09Z

examples/contrib/detection/main.py

+        engine.state.model_outputs.append(engine.state.output)
+
+    @visualizer.on(Events.ITERATION_COMPLETED)
+    def submit_vis_images(engine):


Does this ok in distributed ? I would have added a @one_rank_only() decorator.

I havn't tested it on multi gpu core yet, there is a @one_rank_only() protection on the trainer's epoch_end so the visualizer engine will not be called on other ranks.

louis-she · 2022-04-06T04:46:25Z

@vfdev-5 @sdesrozis
added retinanet support and distributed training, have tested on 2 GPUs.

louis-she · 2022-04-06T07:33:37Z

One of the ci is failed cause codecov -4.27%. I think there are no need to write test code for example right? Should we add example folder in the ignore in codecov.yml ?

louis-she · 2022-04-06T07:36:12Z

I see that there is a PR of mAP implementation here #2130, but there was no update for like half a year.

Can I open a new PR to implemente mAP?

vfdev-5 · 2022-04-06T11:11:28Z

examples/contrib/detection/README.md

+## Usage:
+
+```bash
+python main.py exp_name


It is a bit unclear what is exp_name here. We may do something like

python main.py <exp_name> # for example # python main.py frcnn

vfdev-5 · 2022-04-06T11:13:28Z

examples/contrib/detection/main.py

+    local_rank: int,
+    device: str,
+    experiment_name: str,
+    gpus: Optional[Union[int, List[int], str]] = None,


Looks like gpus is unused ?

examples/contrib/detection/README.md

examples/contrib/detection/main.py

github-actions bot added the examples Examples label Mar 21, 2022

louis-she changed the title ~~add image detection example~~ [WIP] add image detection example Mar 21, 2022

vfdev-5 reviewed Mar 21, 2022

View reviewed changes

sdesrozis reviewed Mar 22, 2022

View reviewed changes

louis-she changed the title ~~[WIP] add image detection example~~ add image detection example Mar 30, 2022

louis-she requested review from sdesrozis and vfdev-5 March 30, 2022 05:10

vfdev-5 reviewed Mar 30, 2022

View reviewed changes

sdesrozis reviewed Apr 3, 2022

View reviewed changes

louis-she force-pushed the master branch from 3663b50 to dd6fe53 Compare April 5, 2022 14:21

louis-she force-pushed the master branch from 862bc84 to d05a893 Compare April 6, 2022 08:41

vfdev-5 reviewed Apr 6, 2022

View reviewed changes

examples/contrib/detection/main.py Outdated Show resolved Hide resolved

louis-she closed this Sep 30, 2022

louis-she force-pushed the master branch from 05dd88f to b3bef8c Compare September 30, 2022 08:12

Uh oh!

Conversation

louis-she commented Mar 21, 2022

Uh oh!

vfdev-5 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sayantan1410 commented Mar 25, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

louis-she commented Apr 6, 2022

Uh oh!

louis-she commented Apr 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

louis-she commented Apr 6, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

louis-she commented Apr 6, 2022 •

edited

Loading