Open AD Kit 2.0 scope

[ghost]

I would like to kick off discussion about what Open AD Kit 2.0 should include, especially from those who are not able to join the regular working group meetings or perhaps do not feel confident enough to voice their opinions.

This topic was discussed during the WG meeting of March 30th/31st, and so I hope to finish this discussion and finalise the v2.0 scope by the second WG meeting in April (13th/14th).

References:

Use-cases and Benefits of the Open AD Kit 2.0 (1st draft).pdf
Open AD Kit Overview.pdf

• Note that the PDFs above have been included here for reference, but are works in progress that are still being updated

[ghost]

Use-cases and Benefits of the Open AD Kit 2.0 (1st draft).pdf

In the previous meeting, I explained the above presentation to provide the starting point to discuss use-cases and benefits that the Open AD Kit version 2.0 will bring to users. To choose appropriate technologies, I think it should be valuable for us to firstly consider the use-cases and benefits, and then we can define the scope of the Open AD Kit version 2.0.

I showed example use-cases and benefits in page 8, and I hope we can discuss and dig into them here.

[ghost]

I got questions from Fatih ( @xmfcx ) in the meeting, and I'll answer them below:

1, 3 and 4 (local deployment only) could be achieved before, so what will v2.0 do differently?

I think that v1.0 achieves some high-level use-cases at some level, but v1.0 does not seem to be sufficient to perform CI/CD or DevOps required in the software defined vehicle (SDV) development. So, v2.0 would provide use-cases and benefits that are demanded in the SDV development. These high-level use-cases can be digged into a little more deeply to appear the difference between v1.0 and v2.0.

what is the difference between 2 and 5?

The no.2 "Simulate scenarios/sensors" means simulations. The no.5 "Debug the system" means that developers can easily prepare the cloud or vehicle-edge environments in their local environment for debugging.

I hope that it answers your questions.
Thank you!

[kenji-miyake]

I think we have to consider the preparation phase such as "Hardware Selection", "Map Creation", etc.
Even though we can't change the HW specs for the Bus ODD, we should document it in a place where people can easily find it.

[armaganarsln]

Hardware selection and architecture is prepared at the beginning of the project for each ODD. The previous ODDs are here: https://github.com/autowarefoundation/autoware-projects/wiki/autonomy-hardware-working-group
https://drive.google.com/open?id=1d3PjkqIypyrm0xwk_AYRP0EGApPxnSw6&authuser=armagan%40leodrive.ai&usp=drive_fs
The current ODD is here and not finalized: https://drive.google.com/open?id=1yGgWudSl04lOhgqgGP-PrNxzVlHH_PMb&authuser=armagan%40leodrive.ai&usp=drive_fs

[NaoyoriTanzawa]

Let's determine the use cases first considering the benefit.
Then, we can discuss which technology given by Kasper's presentation works to achieve the use cases.

@LalithVipulananthan Please give the timeline to close this discussion.

[ghost]

As requested, I have updated the discussion to include my proposed timeline, as well as adding links to PDFs of Hashimoto-san's document and Kasper's document.

[Igata-ctrl]

It might be a good idea to go back to the Vision and Mission of the working group instead of showing the open ADK 1.0 content as a back ground, as v1.0 was more like a simplified feasible study of the containerization.
https://github.com/autowarefoundation/autoware-projects/wiki/Open-AD-Kit-working-group

Secondary, though its'a small issue, the term "Robo-bus" can be misleading as it sounds like a driver-less bus, which ITRI, AWF or any other player in the industry is claiming as far as I know. I'd like to recommend to continue to use "public road bus".

[armaganarsln]

One use case example of Open-AD Kit could be online sensor calibration. We could maybe start with Lidar-Lidar calibration as the first step since bus ODD requires using multiple Lidar. We don't need to do this as an action item but could be in conOps in Hashimoto san presentation.

[kaspermeck-arm]

One of the most important aspects of Open AD Kit is the build, test and deployment infrastructure. The infrastructure is orthogonal to the use-cases, so I don't quite understand what is meant by defining the use-cases first.

Suggestion for the scope of Open AD Kit 2.0

• Bus ODD (this is the microservices application)
  -- simulation in cloud (also for "desktop" development hardware)
  -- live sensor data in vehicle
• Multi stage build pipeline (build, CI/CD)
• Testing, e.g. ROS package test, unit tests (test, CI/CD)
• Public container registry for the build artifacts, i.e., container images
• Integration testing using containers and orchestrator (cloud deployment, CI/CD)
• OTA service (eSync Alliance)
• distro with container runtime and orchestration (EWAOL, edge and cloud)
• CPU and GP-GPU hardware (edge and cloud)

I will work on refining the list above.

[kenji-miyake]

so I don't quite understand what is meant by defining the use-cases first.

I believe what is important to projects is to show the product's value to the stakeholders and involve them.

Use cases are useful for that. It can explain:

• What are the target users and usage of the feature?
• What are the current problems?
• Why can the feature solve the problems? (Is it an appropriate approach?)
• How beneficial is it if the problems are solved?

If stakeholders (including TIER IV) don't understand that, they can't provide resources for the project because they can't judge the ROI.

@kasperornmeck Is this explanation clear for you?

The infrastructure is orthogonal

What did you mean by this? I'm sorry but I couldn't understand well.

[NaoyoriTanzawa]

@kasperornmeck
Thank you for the proposal. I am still having similar questions to Miyake-san's ones, although I joined the Open AD Kit CI/CD bi-weekly sync.
I just checked your slides again. I think the objective can be clarified on the page 2. I understand SOAFEE and SDV are the concept. On the other hand, I assume those should have any objective to be achieved. This is why we are trying to start developing any system with specific use cases.
https://docs.google.com/presentation/d/1aPaW-BkNnFMA-LhVgaxjO3qo5HU-YCqpOtjp-ZJCP2k/edit#slide=id.g1134780ae1b_0_15

[doganulus]

As a researcher in V&V of autonomous systems and a testing tool developer, I would be interested in a containarized Autoware + containerized simulator to research&develop more advanced scenario-based testing and verification tools.

The OpenAD kit would help integrating different testing approaches into the workflow and evaluate them against Autoware on a scalable platform. Currently, setup times from the zero to the first simulation, and from the first to many many simulations is too much effort.

For integration, we would need interfaces to control simulation actors and selectively enable/disable Autoware functions, as well as use ground truth sometimes from the simulator and/or deliberately inject faults to challenge the Autoware implementation harder.

[ghost]

Until now, we have been discussing several candidate technologies, like containerization and microservice architectures. I think some of them are ambitious challenges for vehicle-edge systems that have limited hardware resources. To choice better solutions, I would like to propose the following steps to consider candidate technologies that will be included in version 2.0:

1. Define a use-cases and benefit that the Open AD Kit bring to users of each version (hopefully identify use-cases and benefits of a final version, too)
2. Consider candidate technologies that achieve the use-cases on each version (especially identify them on version 2.0) while considering a project time frame and the feasibility of them

To consider use-cases of the Open AD Kit and identify when we will implement them with specific technologies, I think it is important to create a roadmap based on use-cases and benefits that each version will bring to users. So, I prepared a draft of the roadmap that proposes use-cases and benefits of each version of the Open AD Kit.

I hope we can discuss use-cases, benefits and candidate technologies with the roadmap. If you have any ideas or comments, please feel free to add them to the document or here.

[ghost]

@d-hashimoto The outstanding questions from Kasper above were discussed in today's WG meeting. We attempted to clarify as much as possible based on the roadmap proposal document, but further input from you is needed so that we can finish the discussion and finalise the project scope. 🙇🏾

• 1.1 Easily reproduce vehicle-edge environments
  • The roadmap document talks about a "pre-build container". Could you clarify what is meant by this term?
• 2.1. Simulate sensors
  • For Open AD Kit 1.0, the Planning Simulator is supported locally and in the cloud (using Scenario Runner for cloud CI/CD)
  • For Open AD Kit 2.0, is there a plan to support other simulation types?
    • I asked the Simulation WG in Discord for information on new simulation types, but it would be good to confirm what you had in mind.
• 4.2. Update wirelessly
  • The roadmap's use case is to update container images on vehicle-edge devices, but could the scope be described in more detail? For example, do we want to be able to remotely manage the edge device? Do we want to be able to update edge device firmware as well?
• 5.1. Monitor fleet data
  • How is this done today? What data is gathered and how?
  • Is this just recording of data to rosbags? Is there something different that is planned for 2.0?

[NaoyoriTanzawa]

@kasperornmeck Let me reply to you instead of Hashimoto-san. Sorry for the delay.

1.1 Easily reproduce vehicle-edge environments
This is one of the main goals of EWAOL, ensuring that the deployment environment is consistent with the needs of the applications.

I understand. Thank you for letting us know!

2.1. Simulate sensors
Does replaying a ROSBAG count towards this? Are there other ways of doing this?

We assume a simulator with ROSBAG can handle this.

4.2. Update wirelessly
This can be done in many ways, e.g. for containers, a simple "docker pull", using "docker-compose run", or K3s. Is there anything else you have in mind?

As you mentioned, we understand there are several ways and a solution from eSync is one of the options.

5.1. Monitor fleet data
How is this done today? What data is gathered and how?

This is not achieved with Open AD Kit 1.0.

[NaoyoriTanzawa]

@LalithVipulananthan As we discussed, let me reply to you as follows.

1.1 Easily reproduce vehicle-edge environments
The roadmap document talks about a "pre-build container". Could you clarify what is meant by this term?

It means the container is validated independently.

2.1. Simulate sensors
For Open AD Kit 1.0, the Planning Simulator is supported locally and in the cloud (using Scenario Runner for cloud CI/CD)
For Open AD Kit 2.0, is there a plan to support other simulation types?
I asked the Simulation WG in Discord for information on new simulation types, but it would be good to confirm what you had in mind.

This is under discussion at Simulation WG. The simulator with ROSBAG could be supported this year.

4.2. Update wirelessly
The roadmap's use case is to update container images on vehicle-edge devices, but could the scope be described in more detail? For example, do we want to be able to remotely manage the edge device? Do we want to be able to update edge device firmware as well?

We can involve eSync in this discussion.

5.1. Monitor fleet data
How is this done today? What data is gathered and how?
Is this just recording of data to rosbags? Is there something different that is planned for 2.0?

I think TIER IV can propose this more specifically based on our experience.

[ghost]

@kasperornmeck @LalithVipulananthan
I apologize late reply, and I will answer them:

• 1.1. Easily reproduce vehicle-edge environments

  • It was a typo, and I meant "prebuilt container image."

• 2.1. Simulate sensors

  • For Open AD Kit 2.0, "logging simulator" might be implemented.

• 4.2. Update wirelessly

• To define the scope of OTA, we can gradually expand the scope of OTA, like following steps:

  1. Define hardware components for which software is updated wirelessly (ex: vehicle ECUs, sensors CPU, etc...)
  2. Define target layer that will be updated (ex: application layer, middleware layer, operating system layer, firmware layer)
  3. Define granularity of containerization (this seems to be discussed in the WG on 5/26)

• To narrow the scope of the version 2.0, I can suggest the following example:

  1. Vehicle ECU(s)
  2. Application Layer (only autoware)
  3. Current monolithic image or Splitting out perception
     

• 5.1. Monitor fleet data

  • For updating wirelessly, we need to know the status of the application if we decide the OTA scope like above. If we operate the application as a docker container(s), we need to know the container(s) status and application status remotely. It is not just recording data to rosbags.

[kaspermeck-arm]

@NaoyoriTanzawa @d-hashimoto
Thanks for your responses. It seems like we are aligned now! :)

[ghost]

Since the initial scope for Open AD Kit 2.0 was finalized in this week's Open AD Kit meeting, I will be locking/closing this discussion. If there are any further comments or feedback regarding the current scope, then please discuss in the next WG meeting (June 9th) or comment in the Discord channel between now and that meeting.