TUM Teleoperation Software goes Autoware with EDGAR

[ines29]

TUM Teleoperation Software goes Autoware with EDGAR

@baskKeT @David-Brecht @niklaskrauss-tum @NJGehrke @richypid @timsu-98 @nijik96 @ines29

---

1. Our Goals
   The TUM Teleoperation software aims to provide a modular and open architecture that enables both remote driving and remote assistance within a unified framework. It supports automated vehicles in complex or unexpected situations where autonomy reaches its limits. The system is designed for easy integration across different vehicle and automation platforms through standardized interfaces and adaptable human–machine interfaces. Moreover, it serves as a research platform for developing, testing, and benchmarking teleoperation concepts and technologies.
   

---

2. Software Functions

The software supports Remote Driving and Remote Assistance concepts for different vehicles and automation systems.
In the current open-source release, we have implemented:

• Direct Control – providing low-latency remote vehicle operation with direct actuator access.
• Trajectory Guidance – enabling remote operators to send trajectories to an autonomous vehicle.

Ongoing developments include:

• Perception Modification – allowing operators to correct perception outputs to ensure mission completion and minimize the intervention time & responsibilities of the operator.
• Shared Control – Extending beyond direct control, shared control combines human input with autonomous decision-making to achieve smoother, safer, and more efficient remote driving.
  The architecture follows a ROS 2-based modular structure, supporting distributed deployment, flexible data routing, and real-time telemetry and video streaming.
  

tg_video.mp4

---

4. Deployment to different Vehicles
   The TUM Teleoperation software is designed to be highly portable across different vehicle platforms. Thanks to its ROS 2-based modular architecture and standardized message interfaces, integrating it into a new vehicle typically requires only adapting the Vehicle Interface to the vehicle’s specific control and feedback signals (e.g., throttle, brake, steering, gear). The rest of the teleoperation stack including video streaming, operator interface, and network communication remains unchanged. This design makes it straightforward to deploy the software on various drive-by-wire systems, simulators, or real vehicles with minimal effort.
   This was demonstrated by connecting to a robo-racer vehicle, controlling the vEDGAR simulation, and also controlling vehicles from other universities with our software.
   

---

5. Integration in Autoware

• Problem: Current teleoperation interfaces of Autoware only allow Direct Control of the vehicle and cooperation on planning level via the AD API
• Remote Driving and Assistance concepts like Trajectory Guidance and Perception Modification required additional interfaces
• Trajectory Guidance: Interface to pass an external trajectory to the controller
• Perception Modification: Interface to modify predicted objects and traffic_signals
• Aim: Integrate additional teleoperation interfaces into Autoware with minimal changes of Autoware code and low latency overhead
• Implementation
  • autoware_vehicle_cmd_gate
    • additional subscriber to teleoperation_status
    • adjusted control_cmd forwarding in remote state
  • tum_autoware_teleoperation_interfaces
    • generic_gate nodes to switch between remote and auto topic and forward it to original Autoware topic
    • Modifier to modifiy auto msg if necessary
    • Current interfaces: trajectory, predicted objects and traffic_signals
  • tum_autoware_teleoperation_msgs
    • message and service definitions for the interfaces
  • autoware_launch & autoware.universe/launch
    • Remapping of gate_inputs
    • Launching of the interface nodes
  • TUM Teleoperation vehicle_interface for Autoware
    • Mode changes via AD API
    • Subscription to sensor and automation data
    • Forwards teleoperation signals to Autoware
    • Controls generic_gates and modifier nodes

---

6. Our Team

The project is developed by Dr.-Ing. Diermeyer and a team of 9 PhDs at the Chair of Automotive Engineering (FTM) at the Technical University of Munich (TUM). Our interdisciplinary team combines expertise in:

• RD & RA Concepts
• Robust network & streaming
• Resilience & Safety
• Human–Machine Interaction

---

7. Achievements
   Driving with EDGAR – our research vehicle for autonomous and teleoperated driving and autoware
   The WiesnShuttle project, conducted during last year’s Oktoberfest in Munich, marked one of the first large-scale deployments of our teleoperation system in a real-world urban environment. Building on the experience gained there, this year’s IAA Mobility featured an extended live demonstration of remote assistance concepts, showcasing seamless handover between autonomous and teleoperated driving modes. Currently, the team is working on the Autonomous Driving Licence (ADL) – an ongoing research initiative aimed at defining standardized procedures, safety criteria, and competence requirements for certified teleoperators of autonomous vehicles.

---

8. Software Release
   The open-source repository is available here: https://github.com/TUMFTM/teleoperated_driving
   Reference: Kerbl, Tobias, et al. "TUM Teleoperation: Open Source Software for Remote Driving and Assistance of Automated Vehicles." arXiv preprint arXiv:2506.13933 (2025).

---

9. Further Discussions
   We are currently working on a seamless handover mechanism between Autoware and the TUM Teleoperation system, enabling smooth transitions between autonomous and remote operation. This includes synchronized vehicle state management, safety supervision, and shared control logic to ensure continuous and transparent operation.
   We want to present this with ADL and put our software to the test to assist Automation in passing it.

If you are interested in our software, the latest features, or our upcoming projects, we look forward to your feedback and collaboration!