[Proposal] Integrating V2X as an additional sensor to Autoware : Traffic Light signal sensing use case

[damodarRohit]

Motivation

On behalf of Danlaw, as part of ongoing discussions with API WG, I would like to propose the initial draft of the design for integrating V2X as an additional sensor to existing Autoware stack.

For the initial proposal, the use of V2X Map message and Traffic light signal phase and timing message is considered. This information can be fused with existing image-based traffic light classification. Additionally, V2X can provide remaining time of each signal which can be used by the planning module in the future.

High-level architecture

We would like to propose the following modified architecture for the existing Autoware stack.

V2X Sensing Pipeline

The sensing pipeline handles any data manipulation required for different standards or different V2X OBU hardware/ software stack.

v2x_driver

This module acts as an interface between V2X OBU (on-Board Unit) hardware and Autoware.

• Input topic(s) : /v2x/tx_data
  This topic contains data from Autoware to be broadcasted by the OBU
• Output topic(s) : /v2x/decoded_rx_data
  This topic contains ASN.1 decoded V2X data as received by the OBU

v2x_aw_adapter

This module does any data manipulation required for various standards and converts data to interface with Autoware

• Output topic(s) :
  1. /sensing/v2x/map
  2. /sensing/v2x/spat
  3. /sensing/v2x/tracked_objects
     More topics may be added in future with other identified use cases
• Input topic(s) : Input topics can be data shared by Autoware that may be used by other V2X entities like sensor data or path plan of Ego vehicle.

v2x_map_matching

This module compares V2X MAP message (for ITS standard or similar message in different standards) with Autoware vector map and does a matching of the lanes. This information is then used to match the Traffic light ID used by V2X and the traffic light ID used by Autoware.

When the ego vehicle enters an intersection and receives a MAP message, the module parses through the MAP message to obtain stop line center point or similar attributes of each lane in the V2X MAP message. The Lanelet2 APIs is then used to identify which lane this attribute corresponds to. Once a match is obtained the entry is added to a 'map matching table'.

A timeout mechanism is used to maintain the 'map matching table' entries. For each entry/ RSU ID in the list, a timestamp is appended when the entry is created. Every time a MAP message is received, three scenarios exists:

1. The V2X traffic light ID is not present in the table. In this case, a matching is done and an entry is created.
2. The V2X traffic light ID is present and current_timestamp - entry_timestamp < timeout. In this case, the entry_timestamp is updated with current_timestamp.
3. The V2X traffic light ID is present and current_timestamp - entry_timestamp >= timeout. In this case, a matching is done again and the entry is updated.
4. Any entry in the table with a current_timestamp - entry_timestamp >= timeout is considered INVALID and is removed from the table.

When a SPaT message is received, the 'map matching table' is used to identify the Autoware traffic light ID of each signal ID. Then the signal phase and remaining time values are assigned and published under traffic_light_signal topic to the traffic_light_arbiter module.

The sequence diagram below details the above procedure:

I appreciate comments and inputs on the proposal.

** Edit 1 (2024-03-14) : Updated High-level architecture diagram to remove pending/ future modules under discussion

[mitsudome-r]

What does special_vehicle_planning do in the behavior planner?

[YoshiRi]

I guess this contains something like emergency vehicle avoidance.

[damodarRohit]

@mitsudome-r

This module is related to emergency vehicle avoidance, as @YoshiRi mentioned. An initial draft proposal was created earlier related to this #4055 . The module is a placeholder for future update in planning module. It should analyze information related to emergency vehicle like approach location, lane or similar and provide input to behavior_path_planner to decide an appropriate scenario (lane change, side shift, pull_over, etc.).

However, as per discussion in API WG, traffic light use case was prioritized and the intricate details of the design for 'Obstacles & Emergency vehicle details from V2X' is not completed yet.

Sorry for the confusion. I have now updated the High-level architecture and removed future/ pending components. I am also attaching the specific components applicable to Traffic Light Use case, for reference.

[miursh]

@YoshiRi Could you please take a look at this proposal?

[YoshiRi]

As for the perception object recognition part, the structure seems reasonable.
Of course, Time sync and object merge process should be well considered.

[damodarRohit]

@mitsudome-r I am attaching a sample V2X MAP message in ITS (North American) (SAE J2735) standard as a text file.

MAP Message : MAP message sample.txt

SPAT Message : SPAT message sample.txt

The intersection shown is geographic area near Lat : 35.90302326635433; Long : 139.93409274992874.

The image shows representation of the MAP message intersection on a traditional map satellite image.

[lreiher]

Are there any updates regarding V2X support in Autoware? I'm only aware of tlab-wide/AutowareV2X.

I would like to point anyone interested to our own repository ika-rwth-aachen/etsi_its_messages that contains ROS / ROS 2 support for the standardized ETSI ITS V2X messages. The messages are officially released on the ROS APT package lists. We see a lot of potential in integrating these messages into Autoware, while at the same time keeping message formats as close to the V2X message standards as possible.

There is also an open PR tlab-wide/AutowareV2X#52 for integrating the etsi_its_messages into AutowareV2X, although we as the authors of the etsi_its_messages believe that the architecture can still be simplified.