Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SceneSeg model -- Share test results of real scene data collected by different cameras #25

Open
cyn-liu opened this issue Dec 23, 2024 · 13 comments
Assignees
Labels
testing Benchmarking or system testing

Comments

@cyn-liu
Copy link

cyn-liu commented Dec 23, 2024

Description

@liuXinGangChina mentioned in issue9 that different cameras will be used to collect data from different scenarios for model validation.

1. 2MP HFOV - 30° * 1 leopard-ar0233-nvp2650

Scenarioes:

Site: Urban
Weather: Sunny
Illumination: Night and good lighting
Traffic status: rush hour

test1

2. 2MP HFOV - 120° * 1 leopard-ar0233-nvp2650

Scenarioes:

Site: Urban
Weather: Sunny
Illumination: Night and good lighting
Traffic status: rush hour

test2

To be continued....

@cyn-liu cyn-liu changed the title SceneSeg model -- Share test results of real urban scene data collected by different cameras SceneSeg model -- Share test results of real scene data collected by different cameras Dec 23, 2024
@m-zain-khawaja
Copy link
Collaborator

@cyn-liu - thank you for sharing - excited to see these results on roads the network has never seen before on a country outside of the training data. Is there also a link to the videos somewhere? I also suggest that for the wide-angle camera, to crop the bottom portion of the image with the car bonnet, since during training the bonnet region was specifically cropped out.

@cyn-liu
Copy link
Author

cyn-liu commented Jan 7, 2025

Is there also a link to the videos somewhere?

I have re-edited the link above, and and when you click on the corresponding image will open the video link.

@m-zain-khawaja
Copy link
Collaborator

That's great - thanks for sharing, I really appreciate it. For the wide-angle camera 120-degree, the minor artefacts on the bonnet will go away if the bonnet pixels are cropped out before running inference

@cyn-liu
Copy link
Author

cyn-liu commented Jan 8, 2025

For the wide-angle camera 120-degree, the minor artefacts on the bonnet will go away if the bonnet pixels are cropped out before running inference

Thank you for your suggestion. I will crop the wide-angle 120 degree image data and then running inference,and results will be shared.

@cyn-liu
Copy link
Author

cyn-liu commented Jan 8, 2025

Replaced the original test video with the inference result of the cropped image.

@m-zain-khawaja
Copy link
Collaborator

Thanks so much - I really appreciate it 👍

Comparing both videos - cropped and uncropped, it appears that objects on the far left/right edges of the video are better detected in the uncropped version, however, the artefacts are present on the bonnet. In the cropped version, the bonnet artefacts disappear but sometimes the far/left objects are detected with less accuracy.

So, the best recipe is probably -> run inference on uncropped image -> then crop bonnet in post processing

It's so great to see these real-world automotive camera/mounting data because these implementation recipes can only be learned this way ☺️

@cyn-liu
Copy link
Author

cyn-liu commented Jan 8, 2025

So, the best recipe is probably -> run inference on uncropped image -> then crop bonnet in post processing.

Thank you for your suggestion.

I plan to upload three comparison schemes:

  1. run inference on origin uncropped image
  2. run inference on cropped image
  3. run inference on origin uncropped image and then crop the car bonnet through post-processing.

@m-zain-khawaja
Copy link
Collaborator

Excellent - thank you so much!

@cyn-liu
Copy link
Author

cyn-liu commented Jan 9, 2025

  1. run inference on origin uncropped image.

SceneSeg_test2

  1. run inference on cropped image.

SceneSeg_crop_test2

  1. run inference on origin uncropped image and then crop the car bonnet through post-processing.

SceneSeg_post_crop_test2

@m-zain-khawaja
Copy link
Collaborator

Thank you very much for sharing these comparisons - I really appreciate it

@m-zain-khawaja
Copy link
Collaborator

@cyn-liu - is there a plan to do further data collection and testing?

@liuXinGangChina
Copy link

Hello, Khawaja san,

Latest test result for ScensSeg under foggy weather is released in the discussion channel

Image

Please feel free to leave your comment here

Have a nide day!

心刚

@m-zain-khawaja
Copy link
Collaborator

Thank you very much for your efforts to collect and share this data, it is really appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Benchmarking or system testing
Projects
Development

No branches or pull requests

3 participants