How to Detect Traffic Light State, Sign Types, and Cyclists for Dataset Generation

[AndreaMatteucci]

Hello everyone!
I’m working with Carla to generate a dataset of RGB images and the bounding boxes related to the elements listed below. Currently, by using a combination of the RGB Camera and the Segmentation Camera, I’ve managed to capture (and build the bounding boxes for):

• Vehicles
• People
• Presence of traffic signs
• Presence of traffic lights

However, I would also like to be able to obtain:

• Specific light state of the traffic lights
• Type of traffic signs
• Cyclists and motorcyclists

I should mention that, since my computer is not very new, I’ve been using version 0.9.11, and I’m currently trying to adapt the code to version 0.9.14 (which I can run), so that from there I can try to also obtain the information I mentioned above.

Could you suggest some approaches to also obtain these details? Is it possible to get them still using the Segmentation Camera? Or would it be better to use something else? With both version 0.9.11 and 0.9.14?

To obtain traffic light state, I’ve made a few attempts with the segmentation camera by setting thresholds, but so far I’m still running into some issues.

Thank you very much in advance for your help.

[epochDVKHN]

Hello @AndreaMatteucci
While I haven't used the CARLA RGB/Segmentation cameras for this specific task, I can confirm that CARLA provides direct, "ground-truth" methods to get this information directly from the simulator.

Here is a breakdown of how to directly query the data you're looking for.

Part 1: Getting Bounding Boxes

For Dynamic Actors (Vehicles, Pedestrians):
These actors are spawned by you, so you have a direct reference to them. Their bounding boxes are relative to the actor's own center. To get their true position in the world, you must transform the bounding box using the actor's current transform.

# For a Vehicle
vehicle = world.spawn_actor(blueprint, transform)
vehicle_bbox_relative = vehicle.bounding_box
vehicle_transform = vehicle.get_transform()

# Get the BBox in world coordinates
vehicle_bbox_world = carla.BoundingBox(vehicle_transform.transform(vehicle_bbox_relative.location), vehicle_bbox_relative.extent)
vehicle_bbox_world.rotation = vehicle_transform.rotation # Apply the actor's rotation

# For a Pedestrian (same logic)
pedestrian = world.spawn_actor(blueprint, transform)
pedestrian_bbox_relative = pedestrian.bounding_box
pedestrian_transform = pedestrian.get_transform()
pedestrian_bbox_world = carla.BoundingBox(pedestrian_transform.transform(pedestrian_bbox_relative.location), pedestrian_bbox_relative.extent)
pedestrian_bbox_world.rotation = pedestrian_transform.rotation

For Map Actors (Traffic Signs, Traffic Lights):
These objects are part of the map. You can get their bounding boxes in world space directly by using world.get_level_bbs().

# Get all traffic sign bounding boxes
traffic_signs_bbox_list = world.get_level_bbs(carla.CityObjectLabel.TrafficSigns)

# Get all traffic light bounding boxes
traffic_lights_bbox_list = world.get_level_bbs(carla.CityObjectLabel.TrafficLight)

You might also find world.get_environment_objects() useful, as it returns the carla.EnvironmentObject which includes the id, name, type, transform, and bounding_box.

Part 2: Simulator-based methods for specific requirements

Here are a few simulator-based methods to get the specific details you asked for.

1. Specific light state of the traffic lights
You can either get all lights in the world or get the specific one affecting your vehicle.

# --- Method A: Retrieve and inspect all traffic lights in the world ---
# Fetch all actors whose type_id corresponds to traffic lights
traffic_lights = world.get_actors().filter("traffic.traffic_light")

print(f"--- Found {len(traffic_lights)} Traffic Lights ---")

# Display each light's ID and current state
for light in traffic_lights:
    state = light.get_state()  # Returns carla.TrafficLightState (e.g., Red, Green, Yellow, Off, Unknown)
    print(f"Traffic Light ID={light.id}, State={state.name}")

# --- Method B: Identify the traffic light currently affecting a vehicle ---
# Retrieve the traffic light actor influencing this vehicle (if any)
traffic_light = vehicle.get_traffic_light()

if traffic_light:
    state = traffic_light.get_state()  # Current light state (Red, Yellow, Green)
    print(f"Vehicle ID={vehicle.id} is affected by Traffic Light ID={traffic_light.id}, State={state}")

2. Type of Traffic Signs
You can filter all traffic-related actors with the traffic.* wildcard. The type_id string will tell you exactly what kind of sign it is and you can refine your selection from there.

# --- Method A: Retrieve all speed limit signs in the CARLA world ---
speed_sign_list = world.get_actors().filter("traffic.speed_limit.*")

# Display all detected speed limit signs and their IDs
print(f"--- List of {len(speed_sign_list)} Speed Limit Signs ---")
for sign in speed_sign_list:
    print(f"Sign ID={sign.id}, Type={sign.type_id}")

# --- Method B: Retrieve and classify all traffic signs from map landmarks ---
# Get all map landmarks (includes traffic signs, lights, and other markers)
landmarks = map.get_all_landmarks()

all_traffic_signs = []

# Extract corresponding traffic sign actors from each landmark
for mark in landmarks:
    sign = world.get_traffic_sign(mark)

    # Filter out null entries and traffic lights
    if sign and not sign.type_id.startswith("traffic.traffic_light"):
        all_traffic_signs.append(sign)

# Classify each detected traffic sign by its type
print("--- Classified Traffic Signs ---")
for sign in all_traffic_signs:
    sign_type = sign.type_id

    if "stop" in sign_type:
        category = "STOP"
    elif "speed_limit" in sign_type:
        category = "SPEED LIMIT"
    elif "yield" in sign_type:
        category = "YIELD"
    else:
        category = "OTHER"

    print(f"Sign ID={sign.id}, Category={category}, Type={sign_type}")

3. Cyclists and Motorcyclists
Here is a robust way to get a list of all available blueprints, sorted by their class. We check the blueprint's attributes (number_of_wheels and base_type) to be precise.

# Retrieve all vehicle blueprints available in the CARLA world
blueprint_library = world.get_blueprint_library()
all_vehicle_bps = blueprint_library.filter("vehicle.*")

motorcyclist_bps = []
cyclist_bps = []

for bp in all_vehicle_bps:
    # Filter blueprints that explicitly define the number of wheels
    if bp.has_attribute("number_of_wheels"):
        num_wheels = bp.get_attribute("number_of_wheels").as_int()

        # Consider only 2-wheeled vehicles
        if num_wheels == 2 and bp.has_attribute("base_type"):
            base_type = bp.get_attribute("base_type").as_str()

            # Separate motorcycles and bicycles based on their base_type
            if base_type == "motorcycle":
                motorcyclist_bps.append(bp)
            elif base_type == "bicycle":
                cyclist_bps.append(bp)

# Display all identified motorcycle and bicycle blueprints
print("--- Motorcyclist Blueprints ---")
for bp in motorcyclist_bps:
    print(bp.id)

print("\n--- Cyclist Blueprints ---")
for bp in cyclist_bps:
    print(bp.id)

# Create sets of type_ids for faster lookup
motorcyclist_ids = {bp.id for bp in motorcyclist_bps}
cyclist_ids = {bp.id for bp in cyclist_bps}

# Retrieve all current actors in the CARLA world
all_actors = world.get_actors()

# Identify and print actors that match the 2-wheeled vehicle blueprints
for actor in all_actors:
    if actor.type_id in motorcyclist_ids:
        print(f"Found Motorcycle Actor: ID={actor.id}, Type={actor.type_id}")
    elif actor.type_id in cyclist_ids:
        print(f"Found Bicycle Actor: ID={actor.id}, Type={actor.type_id}")

I think these core functions are available in 0.9.11 and 0.9.14 too — best to cross-check the docs to confirm.
Hope this helps you get the ground-truth data you need! ✅

(PS: You might also want to check out #5508 — it covers a similar use case.)