[Proposal] Dynamic Lanelet2 Loading

[StepTurtle]

Hello, we want to share our proposal on dynamic lanelet2 map loading with you.

Our goal is to dynamically load Lanelet2 maps required for the current driving situation, aiming to avoid time consumption and reduce computational costs in the planning pipeline, while also addressing long-term issues associated with Lanelet2 map projection.

With the method we plan to implement, the components in need will be able to retrieve the entire map in a single request (e.g., mission planner), while those who want to receive the map dynamically will be able to receive it according to the location of the vehicle (e.g. behavior planner).

The main idea of this implementation is to create something similar with differential dynamic point cloud loading in Autoware. You can check the implementation of dynamic point cloud loading here:

• https://github.com/autowarefoundation/autoware.universe/tree/main/map/map_loader
• https://github.com/autowarefoundation/autoware_msgs/tree/main/autoware_map_msgs#getdifferentialpointcloudmapsrv

Check out these links to see the issues we're trying to fix:

• Investigation on using Large Scale Lanelet2 Map in Autoware
• Remove Autoware limitations about map boundaries

We have some progress on parsing lanelet2 maps and loading them with custom lanelet parser, but we did not share any prototype yet. We will share it soon.

---

For the method we aim to develop, we plan to implement the following items (These could be change with different approach):

• Divide the single lanelet2 (.osm) file into multiple files. We can develop a tool for this, or we can use the existing tool according to the needs: https://osmcode.org/osmium-tool/manual.html#creating-geographic-extracts. After the division operation, we will have a metadata file as shown below:

divided_1.osm:
  mgrs_code: "35TPF"
  origin_lat: 42.60854
  origin_lon: 28.2548
divided_2.osm:
  mgrs_code: "35TPF"
  origin_lat: 42.60854
  origin_lon: 28.2548
divided_3.osm:
  mgrs_code: "35TPF"
  origin_lat: 42.60854
  origin_lon: 28.2548

...

• Create a custom lanelet parser under lanelet2_extension to read the divided lanelet2 files and create a single lanelet2 map.

• Create a ROS2 service under https://github.com/autowarefoundation/autoware_msgs/tree/main/autoware_map_msgs/srv to communicate between the map loader and the behavior planner. Here is an example service file:

string[] selected_files

---
std_msgs/Header header

autoware_auto_mapping_msgs/msg/HADMapBin

• Update lanelet2_map_loader to be able to load and publish a single lanelet2 file or multiple lanelet2 files.

• Update behavior planner to request the required lanelet2 map from the map loader with service calls.

---

You will be able to load lanelet with the following scenarios:

Whole area loading: The whole lanelet2 map is loaded at once. This is the current situation, and we plan to use this method in the mission planner.

Differential loading: The lanelet2 map is divided into multiple files and loaded according to the current driving situation. This is the method we plan to develop.

---

Possible Approaches

We are considering three different approaches for differential loading, and we will decide on the most suitable one. If you have any suggestions, we would be happy to hear them.

Approach 1:

• Behavior Planner requests Map server with list of lane ids that they would like to load.
• Map Server reply with map tiles that includes the requested lane ids.
• The most challenging one to implement is this approach.

Approach 2:

• Map Server keeps publishing the map tiles that is within certain distance (e.g. 5 km by 5 km)
• Behavior Planner receives the map and updates their map data.

Approach 3:

• Behavior Planner requests Map server with list of osm file ids that they would like to load.
• Map Server reply with map tiles that includes the requested osm file ids.
• It is most similar to the current implementation of dynamic point cloud loading.
• Behavior Planner should have access to lanelet2 metadata (e.g. osm file id) to request the map server.

---

We would be happy to hear any suggestions or opinions you would like to add.

[StepTurtle]

Hi everyone, I just want to share some updates and demo video:

You can check the current works with following draft PRs.

• feat(map)!: add dynamic lanelet loading parameters autoware_launch#872
• feat(autoware_map_msgs): add msg and srv files releated with dynamic lanelet loading autoware_msgs#81
• feat(lanelet2_extension): add multi osm parser autoware_common#234
• feat(map): add differential lanelet loading autoware_universe#6241
• feat(behavior_path_planner): revise behavior planner to work with dynamic lanelets autoware_universe#6393

And check the demo video: Video Link

• The map changes in the 0:07, 5:40 and 8:07 minutes of the video

• In video, Mission Planner use the whole map as usual and behavior planner subscribes to dynamically loaded map.

• We are sending the service calls manually (right bottom terminal). The planning implementation is not ready yet.

About Map Side:

• The implementation is very similar with differential point cloud loading and we have little work on mapping side (e.g. remove hard-coded part, fix parameters ...). So we are open to feedback on draft PRs.

About Planning Side:

• When we use dynamic lanelet loading in behavior we get some errors like: goal pose is not in map, mission route lanelets cannot find in current map and so on. We chose the final point of the route within the map as the goal pose for a quick solution, aiming to test the effectiveness of our proposal. For now proposal seems work but we don't think the method we used in planning is a good solution. Therefore, I would like to ask for solution suggestions that we can use on the planning side from you. (For that reasons, I did not create draft PR for planning side.)
• Also we have other solution about planning pipeline. Once the mission planned, we can divide the mission smaller sizes. In that way we won't have any goal points or lanelets outside of the dynamically loaded lanelet map.

[yukkysaito]

@StepTurtle
This is a wonderful initiative. Thank you for your efforts.

I have a few questions:

• If a regulatory element associated with a lanelet is located outside the divided area, is it possible to reference the object?
• When loading a new area's lanelet2, does it reconstruct the graph? I am concerned about whether there will be a momentary increase in computational load.

This is separate from the current activity, but relevant, so I wanted to share some information. Currently, lane-level (lanelet2) maps have become essential for routing, which poses a challenge for the development of HD mapless behavior and motion planners, as the mission planners for routing still require lanelet2. We are also considering both lane-level and road-level maps. I plan to start a separate discussion on this topic soon.

[StepTurtle]

Hey, I'm glad for your interest. I will try to answer your questions; I hope you find the responses helpful. If anything is missing, please let me know.

If a regulatory element associated with a lanelet is located outside the divided area, is it possible to reference the object?

When loading maps, for instance, if we take approximately a square with a diagonal of 5 km, the regulatory elements within this area will be included in the map, but those outside of this area will not be included. We had assumed that those outside this area, meaning those at very distant distances, would not affect the behavior planner.

Additionally, if lanelet and the element associated with this lanelet are on separate maps, the relationship between them remains intact when these maps are merged.

---

When loading a new area's lanelet2, does it reconstruct the graph? I am concerned about whether there will be a momentary increase in computational load.

To load new areas, I just add following function in this condition to update planner_data_. I am not sure it is enough update or not but I can simply update map data in behavior planner like this.

planner_manager_->resetRootLanelet(planner_data_);

So far, I have had the opportunity to work with small maps, but I will soon test with larger-sized maps. Currently, there is no delay with small-sized maps, but I am uncertain about the processing cost for larger-sized maps

---

This is separate from the current activity, but relevant, so I wanted to share some information. Currently, lane-level (lanelet2) maps have become essential for routing, which poses a challenge for the development of HD mapless behavior and motion planners, as the mission planners for routing still require lanelet2. We are also considering both lane-level and road-level maps. I plan to start a separate discussion on this topic soon.

Thanks for information, waiting details.

Furthermore, if you have any additional insights and suggestions regarding the planning side or all proposal, feel free to share them anytime. They will be highly valuable from my perspective.

[ataparlar]

@yukkysaito
Thank you for your attention.

You mentioned to using both lane-level and road-level maps. I understand that it is in discussion stage. How the road-level maps will be useful for us? I struggled a bit here. What is the main motivation here to use road level maps?

Best regards
Ata Parlar

[yukkysaito]

@StepTurtle Thanks for your reply 👍
@ataparlar
As a future trend, it is anticipated that autonomous driving systems which do not require pre-existing HD Maps for ADAS applications will emerge. For instance, please take a look at this video by DeepRoute.
https://www.youtube.com/watch?v=d_I9JLyH5VA&t=214s

Such systems will navigate using road-level maps (commonly referred to as SD Maps) while generating detailed information through online HD maps for operation. However, for our navigation, namely mission planning, HD maps (equivalent to Lanelet2) are necessary. Ideally, we would like to conduct navigation in mission planning similar to Google Maps (SD Maps) and use HD Maps for the behavior planner.

[RValner]

@StepTurtle
Is the development of this feature still active?

Best,
Robert

[StepTurtle]

Hello, no it is not.

[RValner]

I see. Is it because of the lack of people/resources or there are alternative solutions for scalability to large maps?

[StepTurtle]

According to what I last remember, there were new ideas on this topic, maybe you can find them from https://github.com/orgs/autowarefoundation/discussions/5409 . But I don't know what the final decision was

[RValner]

Alright, thanks a lot for a quick response!