Feature/gpu async encoding speed enhancement #2169
Open
+2,460
−495
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added FOURCC configuration option to OpenCVCamera and OpenCVCameraConfig for specifying video format. - Implemented _validate_fourcc method to validate and set the camera's FOURCC code. - Updated _configure_capture_settings to apply FOURCC settings before FPS and resolution. - Enhanced camera detection to include default FOURCC code in camera info. - Updated documentation to reflect new FOURCC parameter and its implications on performance.
for more information, see https://pre-commit.ci
- Implemented tests to validate FOURCC configuration and its application in OpenCVCamera. - Added checks for valid FOURCC codes and ensured that invalid codes raise appropriate errors. - Included a test for camera connection functionality using specified FOURCC settings.
for more information, see https://pre-commit.ci
…ithout double prefixing
…dependent features
…encoding, including async and GPU encoding guides, performance measurement tools, and test scripts. This cleanup streamlines the codebase and eliminates redundancy following recent enhancements and optimizations.
- GPU acceleration using NVIDIA NVENC (3-4x speedup) - Async background encoding with configurable workers - Automatic CPU fallback for reliability - Timeout protection to prevent stuck processes - Tested with real robot hardware (SO-101) - Includes test scripts and concise documentation Performance: Non-blocking recording with 3-4x encoding speedup
… to streamline the codebase following recent enhancements. This cleanup eliminates redundancy and focuses on the latest improvements in video encoding features.
- Remove unused variable 'result' in encode_video method - Use shutil.which() to get full path to ffmpeg for security - Fix bandit B607 warning about partial executable path
- Format Python files with ruff - Fix code style and formatting issues - Ensure consistent code formatting across the codebase
- Fix trailing whitespace in GPU_ENCODING_README.md and Python files - Apply ruff format to lerobot_dataset.py and record.py - Ensure all files meet pre-commit formatting standards
- Fix ruff linting issues (16 errors resolved) - Apply pyupgrade syntax updates to Python files - Apply prettier formatting to markdown files - Ensure all files meet pre-commit standards
- Update Python syntax to Python 3.9+ standards - Apply modern Python syntax improvements - Ensure all files meet latest pyupgrade requirements
- Fix remaining 10 ruff linting errors - Apply final pyupgrade syntax updates - Ensure all files meet pre-commit standards completely
- Use --py310-plus flag to match pre-commit configuration - Apply Python 3.10+ syntax updates - Ensure compatibility with pre-commit pyupgrade hook
- Apply remaining ruff fixes to all Python files - Ensure all linting issues are completely resolved - Final step to achieve full pre-commit compliance
Cause workers to wait for all the images in their working directory to be complete.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this does
This PR is a close relative of #1671
I rebased it to the lerobot main branch to make it work with dataset format 3.0
At the time of this writing, this PR is few changes ahead of that one. Please let me know if you would rather use the other one.
I have removed batched encoding completely. it is replaced with this feature.
With async video encoding enabled, the 30 to 40 second encoding wait cited here
#1434
becomes about 5s. If the imagewriter format is changed to JPEG from the default PNG, it becomes 0s.
Additionally If GPU encoding is enabled, or the codec is changed from av1 to h264, CPU usage is drastically reduced.
How it was tested
I created new LeRobotDataset instances with the following combinations of settings and confirmed that the saved datasets could be read by lerobot_train without errors being reported
I did find that causing three episodes to be concatenated in one session fails, but this reported seperated in #2161 and I didn't fix it. I couldn't figure out how. I just avoided it by disabling the concatenation.
How to checkout & try? (for the reviewer)
Follow one of the basic dataset recording tutorials and watch for the reduction in the time taken by the save_episode() call. this PR should only be a performance improvement during recording and should not have any other effect.