Open AD Kit WG meeting 2022/07/28

[ghost]

Administrative

• Meeting recording (Password: q4^8rR+!)
• Previous meeting minutes
• Open AD Kit project board
• Open AD Kit 2.0 schedule

Attendees

Chaired by Armağan

• Adrian Macneil (Foxglove)
• Armağan (Leo Drive)
• asilk
• CJ (AWF)
• Daisuke Hashimoto (T4)
• David W (A3VG)
• Dennis Chen (VicOne)
• Eiji Sekiya (T4)
• Faith (AWF)
• Fumihito Ito (T4)
• Hiroshi Igata (T4)
• Kasper Ornstein Mecklenburg (ARM)
• Khalil
• Lalith Vipulananthan (T4)
• Mitra Shah
• Naoyori Tanzawa (T4)
• Ochi Tatsuo (Trend Micro)
• Ryohsuke Mitsudome (T4)
• Stephen Li (AutoCore)
• Tsigigenet Dessalgn (Marvell)
• Xinyu Wang (T4)
• Yigit Cagatay Kuyu (Karsan)

Agenda

• Introduction of new members
• Announcements
• Discussion topics
  • Foxglove presentation
  • Fatih & Kasper will share an update on their discussion outcomes
• Review of backlog actions

Introduction of new attendees/members

• Ochi Tatsuo (Trend Micro)
• Dennis Chen (Trend Micro/VicOne)
  • VicOne is a new subsidiary of TrendMicro developing cybersecurity solutions for electric vehicles and connected cars
  • Brief explanation about their data flow monitoring solution for Autoware.Auto
  • Will reserve a slot for them to present in more detail in a future meeting

Announcements

• None

Discussion topics

Foxglove presentation

• [add presentation PDF]
• presented by Adrian Macneil
• Foxglove is new to the Autoware community, but a lot of experience with autonomous cars (formerly at Cruise for five years, and his team worked on WebViz a web-based visualization platform for ROS)
• Foxglove is a successor to WebViz, supports ROS1/ROS2 as well as non-ROS data like protobuf, JSON, FlatBuffers etc
• Foxglove Studio
  • Robust and extensible visualization and analysis tool
    • Can be used to visualize any sensor data (Lidar, radar, camera, GPS, IMU), maps etc
  • Library of built-in panels (3D view, maps, plots, raw message inspection)
  • Can create pre-built arrangements to share across teams
  • All built using a web stack of Javascript and HTML - can use the Extension API to create custom panels
  • Can visualize live data, use in a vehicle as a dashboard, can load data from local ROSBAGs or from URL,
  • Foxglove Data Platform (paid product) that enables cloud viewing of uploaded recorded data
  • In-app scripting to transform existing data, turn non-visual messages into visual markers, perform calculations on existing values
  • Available in a web browser or a desktop app for Mac, Windows, Linux
  • Demo
  • Q: Are custom data types and specification visualization of those types in the 3D panel supported?
    • A: Have a variety of supported data types/messages (eg: PointCloud, MarkerArray). ROS messages are supported out of the box, along with Foxglove specific messages.
  • Data storage (Foxglove Data Platform)
    • Customers are recording lots of data (simulations, in-vehicle testing etc) and wanted a way to use all of it simply
    • Foxglove Data Platform takes all that data, stores in your cloud account (Amazon S3, Google Cloud etc) via upload endpoints, organizes and indexes it into a data lake and provides an API and UI that can be used to visualize all your data in Foxglove in the cloud without downloading, download via CLI etc
    • Works with rosbags, ROSBAG 1 and 2.
• MCAP - successor to rosbag.
  • rosbag1 can store lots of different message, including custom formats
  • rosbag2 uses db3 which has shortcomings
    • Inefficient in how data is stored (high latency, susceptible to corruption)
    • Doesn't store custom message definitions (need the exact version of source code that created it to be able to open it)
  • MCAP is designed for robotics data (more on the motivation behind creating MCAP)
• Q: Is a CLI tool available for MCAP?
  • Yes. Provides mcap <-> rosbag conversion. There is also a rosbag2_storage_mcap plugin that allows you to do ros2 bag record/play/info commands directly on mcap files
• Q: For realtime communication with a vehicle, Foxglove only provides one-way communication. Is there going to be a way for Foxglove to natively publish message to the ROS stack?
  • Have to use rosbridge (ROS 1 or 2) or use a ROS 2 native connection (only works on desktop and doesn't support custom messages)
  • ros2 has no way of requesting custom messages from a running robot (ros1 had this feature - could request the custom message definition from the robot, but this was left out from ros2 for some reason. Might get fixed in the next ROS version but that's not due for another 12 months)
  • Some performance issues have been reported with rosbridge. If experienced, please report to Foxglove
• In discussion with Foxglove about potentially joining the Autoware Foundation and co-developing an Autoware extension
  • Could use the Extension API to add a custom extension for Autoware specific visualisations and commands.

Fatih/Kasper discussion outcome

• Fatih and Kasper were talking about how we could build Autoware once and then separate out the built code into production containers that were smaller and easier to deploy
• However, if we tried to launch some of those files/packages, there would be issues:
  • colcon build has a setup bash file that shows where everything is, but this becomes harder if things are taken out of the build folder
• How to do releases better?
  • ROS uses the ROS bloom tool: http://wiki.ros.org/bloom
  • Allows you to build packages and release to the ROS build farm
  • Install via apt install
  • Example: https:/github.com/astuff/pacmod
  • Alternatively, can just add their repository and build their packages locally
  • eg: if someone creates their own Autoware package, they can release in the same way and release their own built debian files
  • Gathers dependencies via aptitude
• Goal is to not interfere with the dev environment, but that built ROS nodes can be extracted out with the correct runtime dependencies, finding a smart way to do this
• Mitsudome: Final goal of Autoware Core was to achieve this, by splitting repositories. There are a lot of base Autoware packages, so what are your thoughts on the Autoware Core/Universe base packages? It would be a large apt binary and maintaining separately would be difficult.
  • Fatih: Read about how ROS build farm works and timeline of releases. ROS is very strict and slow. Submit a package for release and have to wait a month before release.
  • Instead, maintain our own list of packages, be more flexible and be less strict about backwards compatible. Compile each module separately and push to the Autoware repository.
• Mitsudome: So we release package by package for Autoware?
  • Fatih: Need some way to automate this procedure, perhaps by changing the current CI - we have 240 packages!
• Mitsudome: What would be the difference in terms of effort between creating a container
  • Fatih: Difference is to make the container as small as possible. Right now containers are large - if we make a small change, you can theoretically copy that changed package to the container but determining runtime dependencies is hard
  • Mitsudome: Thought this was just a case of running colcon build --up-to-package? Is it not more effort to create a build farm if we want to create a minimal container for multiple packages?
  • Fatih: Depends on our goal. Why are we creating containers in the first place? If anyone wants to run different parts of Autoware in different places, creation of those containers will involve compiling packages multiple times. Doing it this way means building once and then choosing the containers you need
  • Kasper: Moving from development from production, we need to stop deploying the development environment to production -
  • Mitsudome: Not sure if all of this will be available for the MIH demo. Will we be able to set all of that up by that time?
    • Kasper: Do unit tests in the development environment. If we can find a way to take all of the dependencies and run it somewhere that is not in the AW dev environment then that is the first step
    • Fatih: No - not yet familiar with the bloom tool and ROS release process. This is a slow progressing thing - someone could start with a single node and then we work on more as we go along.
    • Mitsudome: Would be easier for Autoware users if we are planning this for the long time.
• David: How does this affect the scope for the ARM DevSummit?
  • Kasper: We need multiple containers available and we don't want to deploy the development environment.
• Armağan: Is this not covered by issue #2454?
  • Kasper: Not quite. This is still building developer environments.

Review of backlog actions / open issues / schedule

• Create separate containers for testing Open AD Kit #2454
  • Not much progress due to problems with basic Autoware launch files
  • This issue shouldn't block anyone else. Can use multiple development containers.
  • Want to arrange a meeting with Fatih and Daniel to arrange details of MIH demo
  • Tanzawa-san mentioned the weekly demo planning meetings - Armağan asked him to invite Fatih
  • However, Kasper noted that the meeting runs on Tuesday mornings Japan time and is just about getting Autoware
  • Kasper says that is not moving us forward since it is still building a developer environment.
  • Fatih: Having these separated out would involve compiling less packages and less dependencies involved, so containers would be smaller.
  • Armağan still wants a meeting for Daniel and Fatih
  • Tanzawa-san will ask Daniel to join the WG meetings regularly.
  • Need to align to make sure that this issue is going to be used in the demo. If not, then we need to stop spending time on it.

Actions

• Ito-san to add the steps required for Verify that Autoware Universe can be run with Open AD Kit 1.0 autoware-projects#32, and to add logs for both old and new yocto versions for Kasper to repro and investigate
• Stephen to ping Shark about updating Verify that Autoware Universe can be run with Open AD Kit 1.0 autoware-projects#32, and also to ask if Shark is still able to do the same tests on a Jetson AGX Xavier
• WG to agree on a cut-off date for the microservice PoC work
• Lalith to add the challenges in containerizing just Perception to Create separate containers for testing Open AD Kit #2454
  • Done!
• Hashimoto-san to create a discussion to get feedback on his Open AD Kit 2.0 use cases
  • https://github.com/orgs/autowarefoundation/discussions/2750
• Armağan met with Foxglove to talk about the alliance discussion
• GDPR compliance was discussed briefly during last week's TSC meeting
• WG to write up a test matrix for Open AD Kit 1.0 and desired hardware, and then create a discussion to seek more testers who have access to that hardware
  • https://github.com/orgs/autowarefoundation/discussions/2738
  • Please add comments/feedback to the discussion!
• Create an autoware.org meeting invitation group for the Open AD Kit WG and add it to each meeting as an invitee
  • New group created: https://groups.google.com/a/autoware.org/g/open-ad-kit-working-group-meeting-invitations/
  • Contacted AWF admin and waiting for an update

Issues

• Integrate Autoware Universe with Open AD Kit and review the results
  • Verify that Autoware Universe can be run with Open AD Kit 1.0 autoware-projects#32
  • Was T4 able to provide Kasper with logs so he can investigate the map display problem with the latest yocto build on AVA?
    • T4 will aim to provide the information by the end of next week.
  • Has AutoCore been able to write up their test report?
    • Stephen to ping Shark about updating the issue, and also to ask if Shark is still able to do the same tests on a Jetson AGX Xavier
• Create separate containers for testing Open AD Kit #2454

Schedule review

• MIH demo updates
  • Another team in MIH will test containers on the AVA platform
• Hashimoto-san ConOps/OpsCon discussion
  • Armağan will share in Discord but need to postpone discussion until the MIH demo requirements discussion has been done.

Actions

• Fatih and Kasper to confirm if Fatih's micro-container PoC work is still going to be used for the demo, or if the work should be abandoned
• Tanzawa-san to ask Daniel to join the WG meetings every week going forwards
• Dennis and Ochi from Trend Micro to present about VicOne in the meeting on 08/11 (two weeks time)