Skip to content

Commit a260045

Browse files
authored
Merge pull request #283 from tokk-nv/dev-lerobot
Copy 99-usb-serial.rules first.
2 parents 0e0b9e3 + cc49c97 commit a260045

File tree

1 file changed

+30
-21
lines changed

1 file changed

+30
-21
lines changed

docs/lerobot.md

Lines changed: 30 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Let's run HuggingFace [`LeRobot`](https://github.com/huggingface/lerobot/) to tr
1212
<span class="blobDarkGreen5">Jetson AGX Orin (32GB)</span>
1313
<span class="blobLightGreen3">Jetson Orin NX (16GB)</span>
1414
<span class="blobLightGreen4">Jetson Orin Nano (8GB)</span><span title="Orin Nano 8GB can run Llava-7b, VILA-7b, and Obsidian-3B">⚠️</span>
15-
15+
1616
2. Running one of the following versions of [JetPack](https://developer.nvidia.com/embedded/jetpack):
1717

1818
<span class="blobPink2">JetPack 6 GA (L4T r36.3)</span> <span class="blobPink1">JetPack 6.1 (L4T r36.4)</span>
@@ -21,13 +21,13 @@ Let's run HuggingFace [`LeRobot`](https://github.com/huggingface/lerobot/) to tr
2121

2222
- `16.5GB` for [`lerobot`](https://hub.docker.com/r/dustynv/lerobot) container image
2323
- Space for models (`>2GB`)
24-
24+
2525
4. Clone and setup [`jetson-containers`](https://github.com/dusty-nv/jetson-containers/blob/master/docs/setup.md){:target="_blank"}:
26-
26+
2727
```bash
2828
git clone https://github.com/dusty-nv/jetson-containers
2929
bash jetson-containers/install.sh
30-
```
30+
```
3131

3232
## Work with Real-World Robots - Before starting containers
3333

@@ -40,9 +40,9 @@ This section gives the guide on how you can work through the LeRobot official ex
4040

4141
`lerobot` is designed to show camera view in windows and playback TTS audio while capturing dataset, so it is more convenient to setup your Jetson with its monitor (and speakers) attached to Jetson.d
4242

43-
### a. Check `jetson-container`'s location
43+
### a. Check `jetson-container`'s location
4444

45-
![alt text](images/lerobot_jetson_ssd.png){: style="height:240px;" align=right}
45+
![alt text](images/lerobot_jetson_ssd.png){: style="height:240px;" align=right}
4646

4747
Through out the course of all the workflows of `lerobot`, we will be generating a lot of data, especially for capturing dataset.
4848

@@ -89,7 +89,7 @@ This is already added to `run.sh` of `jetson-containers`, however, we need to ed
8989
sudo vi /etc/pulse/default.pa
9090
```
9191

92-
Find the section loading `module-native-protomocl-unix` and add `auth-anonymous=1`
92+
Find the section loading `module-native-protomocl-unix` and add `auth-anonymous=1`
9393

9494
```bash
9595
### Load several protocols
@@ -113,7 +113,7 @@ pulseaudio --start
113113
It is more convenient if the lerobot programs can always find the device of leader and follower arm with unique names.
114114

115115
For that, we set an udev rule so that arms always get assigned the same device name as following.<br>
116-
This is first done on Jetson host side.
116+
This is first done on Jetson host side.
117117

118118
- `/dev/ttyACM_kochleader` : Leader arm
119119
- `/dev/ttyACM_kochfollower` : Follower arm
@@ -139,27 +139,36 @@ SUBSYSTEM=="tty", ATTRS{idVendor}=="2f5d", ATTRS{idProduct}=="2202", ATTRS{seria
139139
SUBSYSTEM=="tty", ATTRS{idVendor}=="2f5d", ATTRS{idProduct}=="2202", ATTRS{serial}=="00000000000000000000000000000000", SYMLINK+="ttyACM_kochfollower"
140140
```
141141

142-
Now disconnect the leader arm, and then only connect the follower arm to Jetson.
142+
First copy this under `/etc/udev/rules.d/` (of host).
143+
144+
```bash
145+
sudo cp ./99-usb-serial.rules /etc/udev/rules.d/
146+
```
143147

144-
Repeat the same steps to record the serial to edit the second line of `99-usb-serial.rules` file.
148+
Now disconnect the leader arm, and then only connect the follower arm to Jetson. Record the serial for the follower arm.
145149

146150
```bash
147151
$ ll /dev/serial/by-id/
148152
lrwxrwxrwx 1 root root 13 Sep 24 13:07 usb-ROBOTIS_OpenRB-150_483F88DC50304A46462E3120FF0C081A-if00 -> ../../ttyACM0
149-
$ vi ./packages/robots/lerobot
150153
```
151154

152-
You should have `./99-usb-serial.rules` now looking like this:
155+
Repeat the same steps to record the serial to edit the lines of `/etc/udev/rules.d/99-usb-serial.rules` file.
156+
157+
```bash
158+
$ sudo vi /etc/udev/rules.d/99-usb-serial.rules
159+
```
160+
161+
You should have `/etc/udev/rules.d/99-usb-serial.rules` now looking like this:
153162

154163
```
155164
SUBSYSTEM=="tty", ATTRS{idVendor}=="2f5d", ATTRS{idProduct}=="2202", ATTRS{serial}=="BA98C8C350304A46462E3120FF121B06", SYMLINK+="ttyACM_kochleader"
156165
SUBSYSTEM=="tty", ATTRS{idVendor}=="2f5d", ATTRS{idProduct}=="2202", ATTRS{serial}=="483F88DC50304A46462E3120FF0C081A", SYMLINK+="ttyACM_kochfollower"
157166
```
158167

159-
Finally copy this under `/etc/udev/rules.d/` (of host), and restart Jetson.
168+
Finally make sure the file is under `/etc/udev/rules.d/` (of host), and restart Jetson.
160169

161170
```
162-
sudo cp ./99-usb-serial.rules /etc/udev/rules.d/
171+
sudo ls -l /etc/udev/rules.d/99-usb-serial.rules
163172
sudo reboot
164173
```
165174

@@ -180,7 +189,7 @@ lrwxrwxrwx 1 root root 7 Sep 24 16:13 /dev/ttyACM_kochleader -> ttyACM1
180189

181190
### e. (Optional) CSI cameras
182191

183-
![alt text](images/lerobot_csi_camera.png){: style="height:240px;" align=right}
192+
![alt text](images/lerobot_csi_camera.png){: style="height:240px;" align=right}
184193

185194
If you plan to use CSI cameras (not USB webcams) for data capture, you will use the new `--csi2webcam` options of `jetson-containers`, which exposes V4L2loopback devices that performs like USB webcams (MJPEG) for CSI cameras using Jetson's hardware JPEG encoder.
186195

@@ -265,7 +274,7 @@ pactl set-default-sink [SINK_NAME_OR_INDEX]
265274

266275
You can order the Koch v1.1 kits from ROBOTIS. (*Note: they don't come with 3d printed parts*)
267276

268-
- [Follower arm](https://www.robotis.us/koch-v1-1-low-cost-robot-arm-follower/)
277+
- [Follower arm](https://www.robotis.us/koch-v1-1-low-cost-robot-arm-follower/)
269278
- [Leader arm](https://www.robotis.us/koch-v1-1-low-cost-robot-arm-follower/)
270279

271280
TODO:
@@ -333,7 +342,7 @@ You should operate on ther container's terminal.
333342
Follow the [official document's section](https://github.com/huggingface/lerobot/blob/main/examples/7_get_started_with_real_robot.md#4-train-a-policy-on-your-data).
334343

335344
!!! tip
336-
345+
337346
```bash
338347
wandb login
339348
DATA_DIR=data python lerobot/scripts/train.py \
@@ -351,7 +360,7 @@ Follow the [official document's section](https://github.com/huggingface/lerobot/
351360
If you perform the training on other Jetson or PC, `scp` the outputs directory content back to the orinal Jetson that has the leader and follower arm attached.
352361

353362
```bash
354-
scp -r outputs/train/act_koch_test_01/ <USER>@<IP>:/ssd/jetson-containers/data/lerobot/outputs/train/
363+
scp -r outputs/train/act_koch_test_01/ <USER>@<IP>:/ssd/jetson-containers/data/lerobot/outputs/train/
355364
```
356365

357366
### 5. Evaluate your policy
@@ -389,7 +398,7 @@ Follow the [official document's section](https://github.com/huggingface/lerobot/
389398

390399
![alt text](images/lerobot_visuzalize_dataset_html.png)
391400

392-
If everything goes well, you should see
401+
If everything goes well, you should see
393402

394403
<video controls autoplay muted style="max-width: 960px">
395404
<source src="https://github.com/user-attachments/assets/1ec6e4f0-0f85-4a8a-85c0-f70019f3405b" type="video/mp4">
@@ -442,11 +451,11 @@ jetson-containers run -w /opt/lerobot $(autotag lerobot) \
442451
policy=act \
443452
env=aloha \
444453
env.task=AlohaInsertion-v0 \
445-
dataset_repo_id=lerobot/aloha_sim_insertion_human
454+
dataset_repo_id=lerobot/aloha_sim_insertion_human
446455
```
447456

448457
See [Trossen Robotics](https://www.trossenrobotics.com/aloha-kits) for dual-arm ALOHA kits, and [Robotis](https://www.robotis.us/project-bundles/) for the low-cost [Koch v1.1](https://github.com/jess-moss/koch-v1-1) kit used in the [LeRobot tutorial](https://github.com/huggingface/lerobot/blob/main/examples/7_get_started_with_real_robot.md):
449458

450459
<a href="https://github.com/huggingface/lerobot/blob/main/examples/7_get_started_with_real_robot.md"><img src="images/lerobot_lego.jpg" style="max-width:750px;"></a>
451460
> HuggingFace LeRobot - Get Started with Real-World Robots ([YouTube Playlist](https://www.youtube.com/playlist?list=PLo2EIpI_JMQu5zrDHe4NchRyumF2ynaUN))
452-
461+

0 commit comments

Comments
 (0)