Open AD Kit WG Meeting 2022/06/23

[ghost]

Administrative

• Previous meeting minutes
• Open AD Kit project board
• Open AD Kit 2.0 schedule

Attendees

Chaired by Lalith

• Bonolo M (AWF)
• CJ (AWF)
• Daisuke Hashimoto (T4)
• Daniel Shih (National Taiwan University / T4)
• David W (A3VG)
• Esteve Fernandez (T4)
• Fatih (AWF)
• Fumihito Ito (T4)
• Hiroshi Igata (T4)
• Kenji Miyake (T4)
• Lalith Vipulananthan (T4)
• Naoyori Tanzawa (T4)
• Ryohsuke Mitsudome (T4)
• Stephen Li (Autocore)
• Yoshi Takashima (T4)

Agenda

• Introduction of new members
• Announcements
• Review of backlog actions
• Discussion topics
  • Open AD Kit documentation repositories (Hashimoto)
  • ConsOps/OpsCons for Open AD Kit 2.0 (Hashimoto)

Introduction of new attendees/members

• Daniel Shih
  • Professor at National Taiwan University, joined T4 as a part-time consultant
  • Aim is to help redesign Autoware for real-time operation and to work with partners in Taiwan like ITRI and MIH
  • Also has a research team working on Autoware
• Yoshi Takashima
  • Joined T4 a month and half ago. Came from Amazon and Microsoft before that
  • Managing the development team in T4.

Announcements

• No announcements.

Review of backlog actions / open issues

• Integrate Autoware Universe with Open AD Kit and review the results
  • Verify that Autoware Universe can be run with Open AD Kit 1.0 autoware-projects#32
  • T4 is waiting on Docker images for arm64 and Scenario Simulator from CI/CD team
  • Autocore started testing on the PCU with Scenario Simulator this week
• Document Autoware Universe ROS nodes
  • https://github.com/autowarefoundation/autoware.universe/issues/827
  • Aiming to get a PR submitted to document how to get ROS node information for each stack by the end of the week
• Create an autoware.org meeting invitation group for the Open AD Kit WG and add it to each meeting as an invitee
  • New group created: https://groups.google.com/a/autoware.org/g/open-ad-kit-working-group-meeting-invitations/
• Create separate containers for testing Open AD Kit #2454
  • Fatih has created a branch and just started work, planning to do the following:
    • Decide what nodes are required for each of the planned container
    • Use Docker file to build only the required parts
    • Test using the tutorials

Discussion topics

Open AD Kit documentation repositories

Proposal of the configuration of the document repo for Open AD Kit.pdf

• Open AD Kit repo was created
• All of the information to understand Open AD Kit and system boundaries is complicated!
• Proposal for documentation that should be included for Open AD Kit consists of three main parts:
  • System configuration
    • Describes components that make make up Open AD Kit
    • What kind of hardware can be used with 2.0?
    • Which container images are supported by 2.0?
  • Start guide
    • Explains how to set up and run Autoware on the hardware platforms described in the system configuration part
  • Application example
    • Describes an application example as a reference design
    • Allows users to develop their own application based on the reference design (eg: Bus ODD Demo system)
• Questions
  • David: Under the start guide, are we expecting another box for setting up an edge device or an actual test vehicle
    • Hashimoto: Need to think about the details of what we are going to write. If they want to install to an edge device/hardware, then readers should be able to find the installation procedure.
  • David: On system configuration side, we have "Container Image". Is there an objection to changing the title of the box to "Container Images"
    • Hashimoto: Sure.
• No objections to the proposal were made, so this proposal was accepted.

ConsOps/OpsCons for Open AD Kit 2.0

ConOps and OpsCons of the Open AD Kit 2.0 (1st draft for MIH demo).pdf

• Analysis of MIH demo requirements, focusing on MIH APIs. ConOps and OpsCons are based on this analysis
  • 1. Fleet data
    • Automatically collects fleet data and stores this data in the cloud
  • 2. Enhance capability of perception
    • Developers manually annotate data, make ML algorithms better, integrate changes and test on the cloud
    • After testing, Open AD Kit will remotely update vehicle software
  • 3. OTA updates
    • Can simultaneously update multiple ECUs
  • 4. Interoperate with MIH APIs
    • Sensing API
    • DBW API
    • OTA update API (eSync)
• Questions
  • Daniel: Demo will be done in an indoor environment. In order to show the demo, will probably need to use recorded LiDAR data. Is this planned in the document?
    • Hashimoto: I don't think we will have an actual vehicle.
    • Daniel: No space for a vehicle. Will probably use two AVAs with two displays, with a server (cloud)
    • CJ: For the LiDAR sensor, we were talking about using recorded data. We were not planning to have an actual sensor connected up for the demo?
    • Daniel: Correct. On the demo side, we will be in a booth with four sides which a vehicle would not normally see. That's his recommendation. Could have a LiDAR sensor
    • Fatih: Who has access to the Foxconn LiDAR? Will Foxconn provide recorded data? Will any other sensors be attached
    • Daniel: Requested Foxconn to provide recorded data in PKF format
    • CJ: We don't have to use the Foxconn LiDAR. We can use any LiDAR to show incremental improvements in perception capability.
  • Fatih: Will we be showing anything about the Foxconn LiDAR?
    • Daniel: They may provide a prototype for interoperability testing
    • Fatih: So, we will have two different computers, one for the Open AD Kit demo and one for the LiDAR?
    • Daniel: We could just one computer and switch between them?
    • David: Could we also have a prerecorded video showing the LiDAR on a chassis? Or show a live demo of the LiDAR on a chassis at a different location?
    • CJ: Could we demonstrate a LiDAR in the show not detecting pedestrians and then detect a pedestrian in the show Simulated environment could show the development process to make incremental improvements in the slide (slide 7),
      then the live sensor shows the two versions of the LiDAR. Have we done anything like this in the past with other sensing demos? Like for CES?
      • Lalith: Yes.
    • Lalith: In terms of next steps, we need to actually write out how these will be done in the demo
      • CJ: First, we need to map out the demo according to the hardware we will be using.
      • Daniel: Will we be doing that in this working group?
      • CJ: Yes. AWF wants to reuse this demo at other events, not just MIH. Eg: a workshop for ARM DevSummit - give a technical briefing on Open AD Kit and then go through a demonstration like this (can be a walkthrough or using a prerecorded video). Can be with Kasper and other people involved.
    • David: Are we going with the assumption that this will be based on multiple containers?
      • CJ: That's the assumption, but we need to work out the details. Let's say we go down the path of what Fatih is developing (one container for planning/control and one for everything else). This demo is focused on Perception, so when we go through the demo, what toolchain are we using? Scenario Simulator works with Planning, but T4 has a Log Simulator that works with Perception. However, we need to confirm if Web.Auto can be used with multiple containers. Yoshi, have you confirmed this with the team?
      • Yoshi: Have discussed but haven't summarised the data yet
    • Fatih: I thought we were using a prerecorded rosbag.
      • CJ: But which simulator are you using to test the changes?
      • Fatih: Okay, for Perception, we could do this with traffic light recognition. Original CNN is bad and will fail in the same place, then the second one will recognise traffic lights better. Can show this by replaying the rosbag constantly.
      • CJ: Okay, but as a developer, what tools would you use to validate that change?
      • Fatih: How to quantifiably validate the result? Need to run benchmarks for that. Or visually check.
      • CJ: How would you show improvements in bounding box accuracy in a visual way in a demo?
      • Fatih: Can show this in RViz.
      • CJ: So we can just use RViz and demonstrate this.
    • Fatih: My concern is what will be the host operating system. If you want to showcase Open AD Kit and Foxconn LiDAR, Open AD Kit is supposed to use EWAOL with Yocto. Better to just use a base Ubuntu image, or use two computers?
      • Daniel: We have two AVA computers
      • Fatih: Second computer can be a laptop or
      • David: Can we use a NX or <>
      • Fatih: Does Foxconn expect us to write a driver for their LiDAR, or will they provide one? If we have to develop the driver, then we need pcap files to visualise the data.
      • Daniel: Will check into this about the driver and OS
      • Fatih: How will we showcase Open AD Kit there? I will create two containers that can run on Ubuntu. Will we be using EVAOL? What are the monitoring capabilities of EVAOL? Can we connect a monitor and just use RViz?
      • David: This would be a question for Kasper or someone from ARM. Can Fatih document his questions and we can ping someone from ARM.
    • Lalith: How will we work on the mapping of the demo to hardware?
      • CJ: Who is taking the lead on the definition of the setup and the hardware? Needs to be Daniel and Fatih aligning on this, since Daniel is on the ground and Fatih will be setting the architecture of what is being demonstrated? What is the difference between SOAFEE and EVAOL?
      • Fatih: SOAFEE is an architecture definition. EVAOL uses Yocto to load containers to run software .
        https://ewaol.docs.arm.com/en/v1.0/introduction.html
      • Daniel: https://www.adlinktech.com/Products/Download.ashx?type=MDownload&isDatasheet=yes&file=1967%5cAVA_Developer_Platform-preliminary-datasheet-20220331.pdf
      • Fatih: Biggest problem is that he does not have access to hardware that run EWAOL and no experience with Yocto.
      • David: What do you have available?
      • Fatih: Jetson AGX. Have ordered an Orin but not sure if it's arrived.

Actions

• Fatih to add his questions about EWAOL and Yocto as a comment on this discussion
• WG leads to contact ARM (Matt Spencer) and pass on Fatih's questions

[xmfcx]

I'm working on creation of the containers. And regarding to the demo for MIH, here are my questions/concerns:

In the demo will we use EWAOL or Ubuntu 20.04 as the host operating system?

If we'll use EWAOL, then https://gitlab.com/autowarefoundation/autoware_reference_design/-/blob/main/docs/Appendix/Open-AD-Kit-Start-Guide/index.md#table-of-contents all of these instructions should be followed and modified for the containers of the Autoware Universe.

And I am assuming you'd need specific hardware to perform these actions.

EWAOL and X.Org/X server? monitor support

Reference: https://gitlab.com/autowarefoundation/autoware_reference_design/-/blob/main/docs/Appendix/Open-AD-Kit-Start-Guide/run-autoware.md#run-visialization-and-scenario-simulator-on-your-host-machine

Here the Autoware.Auto is running on the AVA platform or PCU with EWAOL.

But the visualizations are running on the host machine (which I am assuming is a x86-64 architecture. Through the network with DDS I think.

Why was this done this way? Is it possible to have monitor support on EWAOL?

I might have misunderstood the instructions steps too, if so I'm sorry.

Summary

To me, the entire process is very long and out of my expertise (I don't have experience with yocto or BSPs).

Also I don't have access to the specific hardware too, it's so hard if not impossible to develop things like these remotely.

So for now, what I can do is, to show that the containers work on the Ubuntu 20.04 OS.

cc: @LalithVipulananthan @kasperornmeck @kenji-miyake @cjohn66 @HamburgDave @mitsudome-r

[kaspermeck-arm]

@xmfcx
It's possible to add X to enable display output to EWAOL, making it a self-contained demo, i.e., removing the need for a separate computer for the visualization. See https://github.com/m5p3nc3r/meta-ewaol-machine/blob/main/kas/machines/ewaol-avadp-xfce.yml. We have an internal repo which is ported to v1.0 of EWAOL.

This is work which we in Arm can do though, as soon as we have the containers and corresponding launch files. Let's connect more on this!