Technical Questions Regarding Autoware Middleware, Localization, Perception, Planning, and Control

[dIronmanb]

Hello, I’ve been using Autoware extensively in my research projects and truly admire the openness and sophistication of your platform. I deeply appreciate the Autoware community’s continuous effort in developing such a powerful and modular system for autonomous vehicles.

I have been studying Autoware Universe and would like to ask a few detailed technical questions to deepen my understanding:

System information
OS Platform: Linux Ubuntu 22.04
Autoware installed from source (release/v1.0)

1. Middleware

As Autoware operates on ROS 2, which uses DDS as middleware, I’d like to confirm:

• Does Autoware primarily rely on socket-based communication, or does it also provide other inter-process communication methods such as shared memory?
• Additionally, does Autoware’s middleware support V2X (Vehicle-to-Everything) communication, or would that require an external interface?

2. Localization

In the localization pipeline, both ndt_scan_matcher and ekf_localizer use GNSS data as well as odometry information.

• Are these GNSS and EKF results used mainly as initialization inputs for NDT matching, or are they continuously fused throughout operation?
  (Reference diagram: Autoware Overall Node Diagram)

3. Perception

In the Autoware node diagram, camera detection modules exist but camera-based tracking is not explicitly shown.

• Does this mean that tracking is primarily handled by the LiDAR detection and LiDAR tracking modules?

4. Planning

I understand that Autoware’s planning module mainly consists of two high-level scenarios: lane driving and parking.
When running a lane driving scenario:

• It seems that multiple sub-scenarios (e.g., lane keeping, lane change, etc.) may run simultaneously. How does Autoware determine which sub-scenario is the most appropriate at runtime?
• Once a sub-scenario is selected, is the output of this module the selected scenario itself or the generated trajectory?
• At the end of the planning process, does the scenario selector take trajectories from both lane driving and parking scenarios and output one selected trajectory?
• Are the lane driving and parking scenarios executed in parallel by separate nodes?

5. Control

The documentation mentions:
Longitudinal controller: PID
Lateral controller: Pure Pursuit, MPC

• Is there also a controller that simultaneously handles both longitudinal and lateral control, or are they always handled separately?

Thank you very much for your time and for maintaining such an excellent open-source project.

[YamatoAndo]

@dIronmanb Hi,

2. Localization
   Are these GNSS and EKF results used mainly as initialization inputs for NDT matching, or are they continuously fused throughout operation?

By default, Autoware estimates the initial position using GNSS positioning, and performs continuous localization thereafter based on the EKF estimation for NDT matching.

However, if you set regularization.enable to true, the GNSS positioning will also be used for NDT matching during continuous localization.
https://github.com/autowarefoundation/autoware_core/tree/main/localization/autoware_ndt_scan_matcher#regularization

[dIronmanb]

Thank you for your kind reply.

I would like to ask for one additional clarification.

n the node diagram, could you please confirm whether the message pose_twist_fusion_filter/pose_with_covariance_no_yawbias from the ekf_localizer node in the first figure is the same as ekf_pose_with_covariance shown in the second figure?

Furthermore, if the initial pose corresponds to ekf_pose_with_covariance, may I ask whether, according to the node diagram, it receives the body-frame acceleration and linear velocity measurements from the IMU?

[mitsudome-r]

1. Middleware

• Autoware uses CycloneDDS by default so most of the topics use it for topic communication. For large data, we also utilize some other interface such as ROS's intra-process communication within a same node container(example) and agnocast (zero-copy communication) as an option for more efficient communication.
• For V2X communcation, we expect the integrator to create a ROS bridge to convert the hardware specific communication for V2X to be converted into ROS topic to use the information. You can check some discussion posts like this as an example.

2. Localization
   (Already answered by @YamatoAndo )

3. Perception
   For tracking modules, we are using 3d space information rather than doing it in the 2D image. These information could either come from LiDAR detection results, or it could also use camera detection results if you use some latest perception modules like BEVFormer

4. Planning

• We currently have behavior_path_planner and behavior_velocity_planner for behavior planning. These planners can load different modules as a plugins to support different scenes (like lane change and lane keeping). The timing when these plugins are triggered are managed by the planner manager. You can check this documentation to understand how it works.
• If multiple submodules (plugins) are triggered, the modification to the path is done sequential by each plugins rather than each plugins publishing different paths
• Lane driving and parking scenarios are working in parallel by different nodes. (Scenario selector will switch between the two)

5. Control
   There is a node called simple_pure_pursuit that does calculation of both lateral/longitudinal control simultaneously.

[dIronmanb]

Thank you very much for your detailed response !