Privately Owned Vehicle Work Group Meeting - 2024/11/25 - Slot 2

[m-zain-khawaja]

Agenda

SuperDepth

• Review of dataset preparation for SuperDepth neural network including datasets which have been selected for curation, methods used for dataset preparation and progress update on dataset creation.
• Update on literature review for lane detection related to LaneDet development.

Attendees

• @m-zain-khawaja (Senior Tech Lead at AWF and Privately Owned Vehicle Work Group Chair)
• David Walmroth (Co-Chair of OpenADKit Work Group at AWF and US Director of Business and Project Development at PIX Moving)
• Tran Huu Nhat Huy (Masters Robotics Student at Tokyo Institute of Technology)
• Krishna Thallapalli (Technical Specialist at Hexagon Manufacturing Intelligence, India)
• Elvin Abdinli (Masters Informatics at TUM)
• Aadarsh Tiwari (Software Engineer at KPIT)
• Abdulla Ahmadkhan (Undergraduate Researcher at Middle East Technical University)
• Artem Kulyk (Senior Software Engineer at ECARX)
• Chakri Allamsetty (Sr. Vice President, Software Engineering at HEVO leading Automotive EV Charging)
• Sarun Mukdapitak (Undergraduate student at Nagoya University)
• Sohel Mahmud (Senior Computer Vision and Machine Learning Engineer)
• Gabriel Toffanetto (Masters Student at Universidade Estadual de Campinas)

Discussion

• @m-zain-khawaja presented an overview of the PoV use-case since many newcomers joined this work group session
• @m-zain-khawaja then shared the latest updates on developments of the SuperDepth network, specifically related to data curation for training
• the training datasets being used for SuperDepth include simulated data, lidar + monocular camera data, and lidar + stereo vision data
• @m-zain-khawaja showed the input data and ground truth data requirements for the SuperDepth network, including images, depth boundaries, metric heigh-map and metric depth-map
• a code walkthrough was performed of the pipeline to process the raw data from MUAD/UrbanSyn simulator datasets to create the required inputs for the SuperDepth network with a focus on the key functions to create the ground truth depth map, create a depth boundary mask, simulate spare stereo supervision, and create a ground truth height map
• Sohel enquired about the structure of the AutoSeg neural network - whether it was based on Transformers, or CNNs - to which @m-zain-khawaja replied that AutoSeg was a pure conv net, however, it contains computational blocks inspired by self-attention and cross-attention mechanisms leveraged in Transformers
• Artem asked about the compute requirements for running AutoSeg, for example, running on an Nvidia AGX Origin - to which @m-zain-khawaja replied that the current SceneSeg network runs at faster than real-time speeds on a mid-tier laptop, and the goal for this development is for the final AutoSeg Foundation model to be able to run online on hardware associated with OEM deployments for this level of autonomy, and that in partnership with UXL - we hope to make AutoSeg and its sub networks cross platform and able to run across different hardware accelerators.
• David enquired whether a laptop alongside various AI edge computers could be used for running AutoSeg, to which @m-zain-khawaja replied that this should be fairly feasible
• Tran enquired about how intrinsic camera parameters were estimated given that explicit calibration was not available for simulator datasets being used for SuperDepth, including MUAD and UrbanSyn - to which @m-zain-khawaja replied that assuming a perfect simulated camera, the pin-hole camera projection should be the centre of the image, and the smaller dimension of the image was used as the focal length - however, due to sparse supervision, SuperDepth is agnostic to focal length of camera.
• Sohel enquired if SuperDepth could work for cameras at different mounting heights - to which @m-zain-khawaja replied that the dataset contains cameras at different mounting angles and mounting heights so SuperDepth should be able to generalise across such scenarios.
• Artem enquired whether SuperDepth is a monocular system or a binocular stereo-system, to which @m-zain-khawaja replied that it is a hybrid approach, where stereo vision is used for sparse depth supervision, and the monocular pipeline does depth completion and scale coming from stereo depth.
• Tran then shared an updated on the LaneDet literature review covering the FPS performance of various methods, and a review of the accuracy of various methods
• Tran enquired about how we should go about choosing a particular method for implementation in LaneDet, to which @m-zain-khawaja replied that we should pick the simplest method that can be integrated with AutoSeg, whilst retaining good speed and accuracy trade-offs. The goal is not to chase benchmarks or publish papers, but to build a system which is comparable to SOTA and achieves the required performance for our technology development goals, and that we should strive to incorporate the key innovation that helped existing SOTA approaches achieve a high performance - to which Tran agreed.

Zoom Meeting Video Recording

Video Meeting Link

Passcode: v!80*sw@

[liuXinGangChina]

miss this meeting due to mis-understanding of the time-slot， will join discussion next week

[m-zain-khawaja]

Looking forward to meeting next week