Updates to the NuRec documentation to address installation problems (#9386)

MattRoweEAIF · web-flow · commit 19ff9be8f569 · 2025-10-24T15:17:41.000+02:00
* nurec docs update

* changed usdz path
diff --git a/Docs/nvidia_nurec.md b/Docs/nvidia_nurec.md
@@ -2,11 +2,11 @@
 # Use NVIDIA Neural Reconstruction with CARLA
 
 NVIDIA Neural Reconstruction (NuRec) refers to the reconstruction and rendering models and services from NVIDIA that support the seamless ingestion of real-world
-data converted to a simulated environment suitable for training and testing Physical AI Agents, including robotics and autononomous driving systems.
+data converted to a simulated environment suitable for training and testing Physical AI Agents, including robotics and autonomous driving systems.
 
 With NuRec, developers can convert recorded camera and LIDAR data into 3D scenes. NuRec uses multiple AI networks to create
 interactive 3D test environments where developers can modify the scene and see how the world reacts. Developers can change scenarios,
-add synthetic objects, and apply randomizations — such as a child following a bouncing ball into the road — making the initial scenarios
+add synthetic objects, and apply randomization — such as a child following a bouncing ball into the road — making the initial scenarios
 even more challenging. With the NuRec gRPC API, developers can bring rendering services directly to their simulation platform of choice, for example, CARLA.
 
 The NuRec gRPC API serves as a conduit of data and rendering between the CARLA replay and the NuRec container, where the scenes are reconstructed and rendered. You can load pre-trained scenes from the NVIDIA Physical AI Dataset for Autonomous Vehicles and define your scenes using the NuRec gRPC API in a python script (`replay_recording.py`). The diagram below further illustrates the relationship between NuRec and CARLA. 
@@ -35,12 +35,12 @@ To use neural rendering in your CARLA simulations, use the NVIDIA Neural Reconst
 
 ### Prerequisites
 
-Before you get started, make sure you have satisifed the following prerequisites:
+Before you get started, make sure you have satisfied the following prerequisites:
 
 - Ubuntu 22.04
-- [CARLA 0.9.16 or newer package installed](https://carla.readthedocs.io/en/latest/download/) (use the Nightly Build before the 0.9.16 release)
+- [Recent CARLA Nightly build package installed](https://carla.readthedocs.io/en/latest/download/)
 - [CUDA 12.8 or higher](https://developer.nvidia.com/cuda-downloads)
-- Python 3.10+
+- Python 3.10 (Python 3.11 and up not currently supported)
 
 ### Hugging face account
 
@@ -57,9 +57,17 @@ The installation downloads some sample datasets from Hugging Face. To complete t
 
 ## Setup
 
+In the following instructions, a CARLA_ROOT environment variable is used to locate the root directory of the CARLA package you have installed as a prerequisite. You should set this variable in your terminal or add the following line to your `.bashrc` profile:
+
+```sh
+export CARLA_ROOT=/path/to/carla/package
+```
+
+If you prefer not to set the CARLA_ROOT variable, replace `${CARLA_ROOT}` with the path to the CARLA package root folder in the following instructions.
+
 ### Prerequisite installation
 
-The install script will attempt to install the following Ubuntu dependencies. To avoid installation problems, we recommend installing these dependencie before running the NuRec install. 
+The install script will attempt to install the following Ubuntu dependencies. To avoid installation problems, we recommend installing these dependency before running the NuRec install. 
 
 **Docker**: The NuRec tool uses Docker images, therefore you need Docker installed on your system. The following packages are recommended:
 
@@ -108,16 +116,20 @@ source vecarla/bin/activate # Activate the venv
 
 Remember to activate the virtual environment in each new terminal session you open. 
 
-### Run the Installer Script
+!!! note
+    Bear in mind that the virtual environment will be created in the folder where you run this command. In order to source the environment, you must be in the same directory where you ran this command. Once the *venv* is activated, you can then navigate to other directories as required. We recommend creating the *venv* in the `${CARLA_ROOT}` directory or the `${CARLA_ROOT}/PythonAPI/examples/nurec` directory.
+
+### Run the installer Script
 
-To get started quickly and easily with the curated sample set from the [NVIDIA PhysicalAI-Autonomous-Vehicles-NuRec dataset](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles-NuRec), navigate to the CARLA root directory on your machine and run the following launch script:
+To get started quickly and easily with the curated sample set from the [NVIDIA PhysicalAI-Autonomous-Vehicles-NuRec dataset](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles-NuRec), navigate to the CARLA root directory on your machine and run the NuRec installation script:
 
 ```bash
-cd <CARLA_ROOT>
-./PythonAPI/examples/nvidia/nurec/install_nurec.sh
+source vecarla/bin/activate # Activate the venv
+cd ${CARLA_ROOT}/PythonAPI/examples/nvidia/nurec/
+./install_nurec.sh
 ```
 
-The script helps you set your HuggingFace access token (if there isn't one already), sets the required environment variables for the NuRec container, pulls the curated sample dataset from HuggingFace, and installs the required Python packages.
+You will be asked to enter your HuggingFace token on the command line, ensure to have it at hand. The script installs the needed prerequisites and Python packages, sets the required environment variables for the NuRec container and downloads the curated sample dataset from HuggingFace.
 
 The script will install the following Python packages:
 
@@ -133,14 +145,18 @@ The script will install the following Python packages:
 
 ### Example datasets
 
-The NuRec tool can make use of a large collection of pre-trained neural reconstruction datasets, the installer will download the collection from the [NVIDIA PhysicalAI-Autonomous-Vehicles-NuRec dataset on HuggingFace](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles-NuRec). The dataset is ~200Gb in size so ensure that you have adequate hard drive space.
+The NuRec tool can make use of a large collection of pre-trained neural reconstruction datasets, the installer will download the collection from the [NVIDIA PhysicalAI-Autonomous-Vehicles-NuRec dataset on HuggingFace](https://huggingface.co/datasets/nvidia/PhysicalAI-Autonomous-Vehicles-NuRec). **The dataset is 1.52 terabytes in size so you must ensure that you have adequate hard drive space**. The script will download the dataset into a folder named *PhysicalAI-Autonomous-Vehicles-NuRec* in the `${CARLA_ROOT}/PythonAPI/examples/nvidia/nurec/` directory.
+
+!!! note
+    If you have previously run the installation script and encountered a problem, you may need to move or delete the *PhysicalAI-Autonomous-Vehicles-NuRec* folder. The script will assume the data is previously downloaded if this folder already exists.
 
 ### Python environment 
 
-If you are using a specific Python environtment, you should install the required Python packages by running the following command from the CARLA directory on your machine:  
+If you are using a specific Python environment, you should install the required Python packages by running the following command:  
 
-```
-pip install -r requirements.txt
+```sh
+source vecarla/bin/activate # Activate the venv
+python3 -m pip install -r ${CARLA_ROOT}/PythonAPI/carla/requirements.txt
 ```
 
 ### Set up your environment variables 
@@ -159,25 +175,20 @@ The replay script uses two environment variables — `NUREC_IMAGE` and `CUDA_VIS
 
 ## Run the CARLA NuRec Replays
 
-**1. Start the CARLA Server:**  From the directory where your CARLA package exists, run the following command:  
+**1. Launch the CARLA Server:**  From the directory where your CARLA package exists, run the following command:  
 
-```
+```sh
+cd ${CARLA_ROOT}
 ./CarlaUE4.sh
 ```
 
-**2. Replay a NuRec Scenario:** Once the CARLA server is running, open a new terminal window and navigate to the directory where your CARLA package exists, then replay a NuRec scenario with the `example_nurec_replay_save_images.py` script. We recommend using the NuRec version 25.07 datasets, which you will find in the `CARLA_ROOT/PythonAPI/examples/nvidia/nurec/PhysicalAI-Autonomous-Vehicles-NuRec/sample_set/25.07_release` directory.
+**2. Replay a NuRec Scenario:** Once the CARLA server is running, open a new terminal window and navigate to the directory where your CARLA package exists, then replay a NuRec scenario with the `example_nurec_replay_save_images.py` script. We recommend using the NuRec version 25.07 datasets, which you will find in the `${CARLA_ROOT}/PythonAPI/examples/nvidia/nurec/PhysicalAI-Autonomous-Vehicles-NuRec/sample_set/25.07_release` directory. Choose one of the example USD datasets from the *PhysicalAI-Autonomous-Vehicles-NuRec* directory to experiment with and copy the path and filename for the following command. To run the example, insert the path and filename of the example you chose into the following command and execute it:
 
-* **Multi-camera replay:** The example script provides a complete, multi-view visualization system, ideal for understanding how to integrate various camera types and create comprehensive monitoring setups. When you run it, it replays simulations with multiple NuRec cameras (front, left cross, right cross) in different camera positions in a Pygame display grid. It also supports additional perspectives pulled from standard CARLA cameras attached to the ego vehicle and multiple camera feeds with different framerates and resolutions. 
-
-!!! note
-    If you are using a virtual environment for Python version consistency, remember to activate the virtual environment with: `source vecarla/bin/activate`
 
 ```sh
 source vecarla/bin/activate # Omit if you are not using a venv
-cd PythonAPI/examples/nvidia/
-python example_nurec_replay_save_images.py --usdz-filename \
-PhysicalAI-Autonomous-Vehicles-NuRec/sample_set/25.07 \ 
-_release/026d6a39-bd8f-4175-bc61-fe50ed0403a3/026d6a39-bd8f-4175-bc61-fe50ed0403a3.usdz #--move-spectator --saveimages
+cd ${CARLA_ROOT}/PythonAPI/examples/nvidia/nurec/
+python3 example_nurec_replay_save_images.py --usdz-filename <path_to_example>/<usd_filename>.usdz #--move-spectator --saveimages
 
 ```
 
@@ -187,7 +198,9 @@ _release/026d6a39-bd8f-4175-bc61-fe50ed0403a3/026d6a39-bd8f-4175-bc61-fe50ed0403
     * `--saveimages`: images rendered by NuRec cameras and CARLA cameras will be saved in a default directory named `data` created in the execution location of the script
     * `--output-dir`: specify the directory for the output images when using the `--saveimages` flag
 
-* **Custom camera parameters:** If you need to replicate specific camera hardware or match real-world camera calibrations, you can specify the camera configuration in the YAML configuration file. You will find the camera configurations in the `carla_example_camera_config.yml` file in the same directory as the example scripts. Modify this file and re-launch the example script above. You can change the target YAML file on line 173 of the `example_nurec_replay_save_images.py` script:
+* **Multi-camera replay:** The example script provides a complete, multi-view visualization system, ideal for understanding how to integrate various camera types and create comprehensive monitoring setups. When you run it, it replays simulations with multiple NuRec cameras (front, left cross, right cross) in different camera positions in a Pygame display grid. It also supports additional perspectives pulled from standard CARLA cameras attached to the ego vehicle and multiple camera feeds with different framerates and resolutions. 
+
+* **Custom camera parameters:** If you want to replicate specific camera hardware or experiment with different camera configurations, you can specify the camera configuration in the YAML configuration file. You will find the camera configurations in the `carla_example_camera_config.yml` file in the same directory as the example scripts. Modify this file and re-launch the example script above. You can change the target YAML file on line 173 of the `example_nurec_replay_save_images.py` script:
 
 ```py
     # Add cameras using the new flexible add_camera method
