setup robot abstractions and teleop #2080
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR sets up robot abstractions and teleoperation for a Meca500 robot with Phantom Omni haptic device integration. It implements the necessary components for data collection, policy inference, and teleoperation in the LeRobot framework.
- Implements OmniTeleoperator class for Phantom Omni haptic device control with position/rotation deltas and button handling
- Adds Meca robot implementation with proper observation/action features, camera integration, and motion control
- Creates collection, replay, and inference scripts for the Meca500 robot workflow
Reviewed Changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/lerobot/teleoperators/omni/omni.py | Main teleoperator class implementing the Omni haptic device interface |
| src/lerobot/teleoperators/omni/controller.py | Low-level controller for haptic device communication and state management |
| src/lerobot/scripts/lerobot_record.py | Adds debug print statement for robot actions |
| src/lerobot/robots/meca/run.py | Simple teleoperation demo script |
| src/lerobot/robots/meca/replay.py | Dataset replay functionality for recorded episodes |
| src/lerobot/robots/meca/mecaconfig.py | Robot configuration with camera and network settings |
| src/lerobot/robots/meca/meca.py | Main robot implementation with observation/action handling |
| src/lerobot/robots/meca/inference.py | Policy inference and evaluation recording script |
| src/lerobot/robots/meca/collect.py | Data collection script using teleoperation |
| cameras.py | Standalone camera testing utility |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| @RobotConfig.register_subclass("meca") | ||
| @dataclass |
There was a problem hiding this comment.
The @RobotConfig.register_subclass("meca") decorator and @dataclass decorator are duplicated. Remove the duplicate decorators on lines 9-10.
| @RobotConfig.register_subclass("meca") | |
| @dataclass |
Copilot uses AI. Check for mistakes.
| index_or_path=3, | ||
| width=640, | ||
| height=360, | ||
| fps=260, |
There was a problem hiding this comment.
The FPS value of 260 for cam_2 is unusually high and may cause performance issues or hardware limitations. Consider using a more standard FPS value like 30 or 60.
| fps=260, | |
| fps=30, |
Copilot uses AI. Check for mistakes.
| def close(self): | ||
| self.device.close() | ||
|
|
||
| device = HapticDevice(callback=device_callback, scheduler_type="async") |
There was a problem hiding this comment.
This global device instance is unused and creates a potential resource leak. The Controller class already manages its own device instance. Remove this line.
| device = HapticDevice(callback=device_callback, scheduler_type="async") |
Copilot uses AI. Check for mistakes.
src/lerobot/robots/meca/meca.py
Outdated
| cv2.imshow("flipped", processed[cam]) | ||
| cv2.waitKey(1) | ||
|
|
||
|
|
||
|
|
There was a problem hiding this comment.
Debug visualization code should be removed from production. These lines will display a window during normal operation which is not intended behavior.
| cv2.imshow("flipped", processed[cam]) | |
| cv2.waitKey(1) | |
| pass |
Copilot uses AI. Check for mistakes.
| from pathlib import Path | ||
| from pprint import pformat | ||
| from typing import Any | ||
| import cv2 |
There was a problem hiding this comment.
The cv2 import is unused in this file. The only change is adding a print statement that doesn't use cv2.
| import cv2 |
Copilot uses AI. Check for mistakes.
2 participants
What this does
Explain what this PR does. Feel free to tag your PR with the appropriate label(s).
Examples:
How it was tested
Explain/show how you tested your changes.
Examples:
test_somethingintests/test_stuff.py.new_featureand checked that training converges with policy X on dataset/environment Y.some_function, it now runs X times faster than previously.How to checkout & try? (for the reviewer)
Provide a simple way for the reviewer to try out your changes.
Examples:
SECTION TO REMOVE BEFORE SUBMITTING YOUR PR
Note: Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR. Try to avoid tagging more than 3 people.
Note: Before submitting this PR, please read the contributor guideline.