feat(vehicle): new package of generic_value converter and generic_value calibrator

[Owen-Liuyuxuan]

Description

This PR introduces two new packages for generic value calibration and conversion, enabling calibration and control of arbitrary float64 actuators beyond traditional throttle/brake pedal systems.

New Packages

1. autoware_generic_value_calibrator: Generic calibration node that learns the mapping between arbitrary float64 input values and vehicle acceleration using Recursive Least Squares (RLS) algorithm.

2. autoware_generic_value_converter: Generic conversion node that converts acceleration commands to arbitrary float64 output values using bilinear interpolation on a calibrated velocity-acceleration mapping table.

Key Features

• Generic Input/Output: Works with tier4_debug_msgs::msg::Float64Stamped instead of vehicle-specific messages
• Flexible Calibration: Supports custom input value ranges (not limited to pedal values)
• Topic Converter: Python utility to convert common message types (Control, Odometry, etc.) to Float64Stamped
• RLS Algorithm: Online learning with two update methods (per-cell or global offset)
• Real-time Visualization: RViz occupancy grid visualization and SVG plot generation
• Comprehensive Filtering: Automatic filtering of invalid data (steering, pitch, jerk, etc.)

Architecture Overview

graph TB
    subgraph "Calibration Phase"
        A[Control Command] -->|topic_converter.py| B[Float64Stamped]
        B --> C[Generic Value Calibrator]
        D[Velocity Report] --> C
        E[Steering Report] --> C
        C -->|RLS Algorithm| F[Calibrated Map CSV]
    end
    
    subgraph "Runtime Phase"
        G[Control Command] --> H[Generic Value Converter]
        I[Odometry] --> H
        F --> H
        H --> J[Float64Stamped Output]
        J --> K[Custom Actuator]
    end
    
    style C fill:#e1f5ff
    style H fill:#e1f5ff
    style F fill:#fff4e1

Loading

Data Flow

sequenceDiagram
    participant Controller
    participant Converter
    participant Actuator
    participant Vehicle
    
    Controller->>Converter: Control Command<br/>(acceleration)
    Converter->>Converter: Load CSV Map
    Converter->>Converter: Bilinear Interpolation<br/>(acc, vel) → value
    Converter->>Actuator: Float64Stamped<br/>(output value)
    Actuator->>Vehicle: Custom Control Signal
    Vehicle->>Converter: Odometry (velocity)

Loading

Calibration Workflow

flowchart TD
    A[Start Calibration] --> B[Load Default Map]
    B --> C[Collect Data]
    C --> D{Data Valid?}
    D -->|No| E[Filter Out]
    D -->|Yes| F[Calculate Acceleration]
    F --> G[Apply Pitch Compensation]
    G --> H[RLS Update]
    H --> I{Update Method?}
    I -->|Per Cell| J[Update Cell Offset]
    I -->|Global| K[Update Global Offset]
    J --> L[Maintain Consistency]
    K --> L
    L --> M[Calculate RMSE]
    M --> N{Error Improved?}
    N -->|Yes| O[Save CSV Map]
    N -->|No| C
    O --> P[End]
    E --> C

Loading

Comparison with Existing Packages

Feature	accel_brake_map_calibrator	generic_value_calibrator
Input Type	Throttle/Brake pedal	Arbitrary float64
Input Message	ActuationCommandStamped	Float64Stamped
Number of Maps	2 (throttle + brake)	1 (unified)
Use Case	Standard vehicles	Custom actuators

Feature	raw_vehicle_cmd_converter	generic_value_converter
Output Type	Throttle/Brake pedal	Arbitrary float64
Output Message	ActuationCommandStamped	Float64Stamped
Number of Maps	2 (throttle + brake)	1 (unified)
Use Case	Vehicle control	Generic mapping

Related links

Parent Issue:

• Link to parent issue/feature request

How was this PR tested?

Unit Tests

Converter Package:

• ✅ CSV loading and validation (test_csv_loader.cpp)
• ✅ Bilinear interpolation (test_value_map.cpp)
• ✅ Boundary condition handling
• ✅ Empty map error handling

Calibrator Package:

• ✅ Core algorithms (nearest search, RLS, lowpass) (test_calibrator_utils.cpp)
• ✅ Python utilities (average filter, 2D map creation) (test_calc_utils.py)

Integration Testing

• ✅ Launch files tested with default parameters
• ✅ Topic converter tested with various message types
• ✅ CSV map generation and loading verified
• ✅ RViz visualization confirmed working

Manual Testing

• ✅ Calibration workflow tested with simulated data
• ✅ Converter tested with generated maps
• ✅ Topic converter tested with Control, Odometry, and TwistStamped messages
• ✅ Edge cases tested (empty maps, boundary values, invalid data)

Notes for reviewers

Key Design Decisions

1. Generic Float64Stamped Interface: Chose Float64Stamped over vehicle-specific messages to maximize flexibility for custom actuators.

2. Two Update Methods:

   • update_offset_each_cell: Higher accuracy, requires more data
   • update_offset_total: Faster convergence, works with limited data

3. Topic Converter: Python script for easy integration without code changes, supporting common Autoware message types.

4. Default Map Generation: Simple linear relationship (accel = value) as starting point, updated by RLS during calibration.

Code Structure

• Converter: C++ node with ValueMap class for interpolation
• Calibrator: C++ node with RLS algorithm and data filtering
• Topic Converter: Python script with extensible conversion map
• Visualization: Python server for SVG plot generation

Potential Future Enhancements

• Support for multiple output values (vector output)
• Adaptive forgetting factor based on data quality
• Integration with Autoware's diagnostic system
• Support for non-monotonic maps (with validation)

Interface changes

Topic changes

Additions

Change type	Topic Type	Topic Name	Message Type	Description
Added	Sub	/generic_value_converter/input/control_cmd	autoware_control_msgs::msg::Control	Control command with desired acceleration
Added	Sub	/generic_value_converter/input/odometry	nav_msgs::msg::Odometry	Odometry for current velocity
Added	Pub	/generic_value_converter/output/value	tier4_debug_msgs::msg::Float64Stamped	Converted float64 output value
Added	Sub	/generic_value_calibrator/input/value	tier4_debug_msgs::msg::Float64Stamped	Generic float64 input value for calibration
Added	Sub	/generic_value_calibrator/input/velocity	autoware_vehicle_msgs::msg::VelocityReport	Vehicle velocity information
Added	Sub	/generic_value_calibrator/input/steer	autoware_vehicle_msgs::msg::SteeringReport	Steering angle information
Added	Pub	/generic_value_calibrator/output/update_suggest	std_msgs::msg::Bool	Flag suggesting map update
Added	Pub	/generic_value_calibrator/output/current_map_error	tier4_debug_msgs::msg::Float64Stamped	Current map RMSE error
Added	Pub	/generic_value_calibrator/output/updated_map_error	tier4_debug_msgs::msg::Float64Stamped	Updated map RMSE error
Added	Pub	/generic_value_calibrator/output/map_error_ratio	tier4_debug_msgs::msg::Float64Stamped	Error ratio (updated/original)
Added	Pub	/generic_value_calibrator/debug/original_occ_map	nav_msgs::msg::OccupancyGrid	Original default map visualization
Added	Pub	/generic_value_calibrator/debug/update_occ_map	nav_msgs::msg::OccupancyGrid	Calibrated map visualization
Added	Pub	/generic_value_calibrator/debug/data_count_occ_map	nav_msgs::msg::OccupancyGrid	Data count per cell
Added	Pub	/generic_value_calibrator/debug/data_count_self_pose_occ_map	nav_msgs::msg::OccupancyGrid	Data count with current position
Added	Pub	/generic_value_calibrator/debug/data_average_occ_map	nav_msgs::msg::OccupancyGrid	Average acceleration per cell
Added	Pub	/generic_value_calibrator/debug/data_std_dev_occ_map	nav_msgs::msg::OccupancyGrid	Standard deviation per cell
Added	Pub	/generic_value_calibrator/debug/occ_index	visualization_msgs::msg::MarkerArray	Velocity and value index labels

ROS Parameter Changes

Additions - Generic Value Converter

Change type	Parameter Name	Type	Default Value	Description
Added	csv_path_value_map	string	$(find-pkg-share autoware_generic_value_converter)/data/value_map.csv	Path to mapping CSV file
Added	max_value	double	3.0	Maximum output value
Added	min_value	double	-5.0	Minimum output value
Added	convert_value_cmd	bool	true	Whether to perform conversion
Added	use_value_ff	bool	true	Whether to use feedforward (lookup table)

Additions - Generic Value Calibrator

Change type	Parameter Name	Type	Default Value	Description
Added	update_hz	double	10.0	Update frequency (Hz)
Added	update_method	string	"update_offset_each_cell"	Update algorithm ("update_offset_each_cell" or "update_offset_total")
Added	get_pitch_method	string	"tf"	Method to get pitch angle ("tf" or "none")
Added	progress_file_output	bool	false	Flag to output log file for detailed calibration data
Added	csv_default_map_dir	string	""	Default map directory (if empty, uses parameters below)
Added	csv_calibrated_map_dir	string	"$(env HOME)/autoware_map_calibration"	Calibrated map directory where output CSV will be saved
Added	output_log_file	string	""	Path to log file for detailed calibration data
Added	precision	int	3	Decimal precision for CSV output
Added	value_min	double	-5.0	Minimum input value range
Added	value_max	double	3.0	Maximum input value range
Added	value_num	int	11	Number of points in input value range
Added	velocity_min	double	0.0	Minimum velocity range (m/s)
Added	velocity_max	double	20.0	Maximum velocity range (m/s)
Added	velocity_num	int	11	Number of points in velocity range
Added	initial_covariance	double	0.05	Initial covariance for RLS algorithm
Added	velocity_min_threshold	double	0.1	Minimum velocity threshold
Added	velocity_diff_threshold	double	0.556	Velocity difference threshold
Added	value_diff_threshold	double	0.03	Value difference threshold
Added	max_steer_threshold	double	0.3	Maximum steering angle threshold
Added	max_pitch_threshold	double	0.02	Maximum pitch angle threshold
Added	max_jerk_threshold	double	1.0	Maximum jerk threshold
Added	value_velocity_thresh	double	20.0	Value-velocity threshold
Added	max_accel	double	5.0	Maximum acceleration limit (m/s²)
Added	min_accel	double	-5.0	Minimum acceleration limit (m/s²)
Added	value_to_accel_delay	double	0.3	Delay from input value to acceleration (s)
Added	update_suggest_thresh	double	0.7	Threshold for update suggestion (RMSE ratio threshold)
Added	max_data_count	int	200	Maximum number of data points per cell

Effects on system behavior

New Capabilities

1. Custom Actuator Support: Enables calibration and control of non-standard actuators (electric motors, hydraulic systems, etc.) without modifying core Autoware code.

2. Flexible Calibration: Supports arbitrary input value ranges, not limited to traditional throttle/brake pedal values.

3. Easy Integration: Topic converter allows using existing Autoware messages (Control, Odometry) without code changes.

No Breaking Changes

• ✅ No modifications to existing packages
• ✅ No changes to existing topics or parameters
• ✅ Fully backward compatible
• ✅ Optional packages (can be used independently)

Integration Points

• Input: Can subscribe to standard Autoware control commands via topic converter
• Output: Publishes generic Float64Stamped for custom actuator interfaces
• Visualization: Integrates with RViz for real-time monitoring
• File I/O: Saves calibrated maps to CSV format compatible with converter

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[codecov]

Codecov Report

❌ Patch coverage is 12.24490% with 1118 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.76%. Comparing base (ce43717) to head (607dd3e).

Files with missing lines	Patch %	Lines
...e_calibrator/src/generic_value_calibrator_node.cpp	0.00%	618 Missing ⚠️
...lue_calibrator/scripts/generic_value_map_server.py	0.00%	178 Missing ⚠️
...eneric_value_calibrator/scripts/topic_converter.py	0.00%	126 Missing ⚠️
...toware_generic_value_calibrator/scripts/plotter.py	0.00%	60 Missing ⚠️
...lue_converter/src/generic_value_converter_node.cpp	2.22%	44 Missing ⚠️
...are_generic_value_calibrator/scripts/csv_reader.py	0.00%	41 Missing ⚠️
...utoware_generic_value_calibrator/scripts/config.py	0.00%	34 Missing ⚠️
...autoware_generic_value_converter/src/value_map.cpp	78.04%	4 Missing and 5 partials ⚠️
...are_generic_value_calibrator/scripts/calc_utils.py	96.70%	1 Missing and 2 partials ⚠️
...utoware_generic_value_converter/src/csv_loader.cpp	91.66%	0 Missing and 3 partials ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #11630      +/-   ##
==========================================
- Coverage   17.84%   17.76%   -0.08%     
==========================================
  Files        1744     1757      +13     
  Lines      121738   123118    +1380     
  Branches    42774    43066     +292     
==========================================
+ Hits        21721    21877     +156     
- Misses      81812    83026    +1214     
- Partials    18205    18215      +10     

Flag	Coverage Δ		*Carryforward flag
daily	20.02% <ø> (ø)		Carriedforward from d9f1102
daily-cuda	17.84% <ø> (ø)		Carriedforward from d9f1102
differential	7.16% <12.24%> (?)
total-cuda	17.84% <ø> (ø)		Carriedforward from d9f1102

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Owen-Liuyuxuan]

Additional Note to reviewers

• It will be great if we can merge this strategy into the throttle/brake calibration packages because many things are shared.
• However, the current implementation is good for acceleration/torque mapping, which will be very useful when the vehicle's dynamics are actually different from its description.

Additional Note to users before PR merge.

These two packages are rather independent, so we can simply grasp the two package folders and build/install them in our local autoware environment.