Skip to content

Conversation

pkooij
Copy link
Member

@pkooij pkooij commented Oct 8, 2025

This PR introduces a way to generate image and text embeddings to a dataset to be more efficient when training a dataset for multiple epochs. For example for learning a general reward we contain a specific dataset with OXE to improve generalization. In order to nor recompute the image and text embeddings each time we finetune for OXE we can use this script to add the embeddings to the dataset. We can additionally remove the videos in the dataset to safe space.

Testing:
Both the generate and validate script were tested on this dataset: lerobot/utokyo_xarm_bimanual. The generated dataset can be found here: pepijn223/utokyo_xarm_bimanual_embeddings.

@pkooij pkooij self-assigned this Oct 8, 2025
@pkooij pkooij added dataset Issues regarding data inputs, processing, or datasets performance Issues aimed at improving speed or resource usage policies Items related to robot policies labels Oct 8, 2025
@pkooij pkooij marked this pull request as ready for review October 8, 2025 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dataset Issues regarding data inputs, processing, or datasets performance Issues aimed at improving speed or resource usage policies Items related to robot policies

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant