CPS Project

This repository contains source code, resources, documentation and final results of the project for the Cyber-Physical Systems course of Automation Engineering at the Univerity of Bologna for the academic year 2023/2024.

Software dependecies & Setup

To run the code inside the repo you need to have Docker installed on your machine. The code has been tested with the version 26.1.4 (build 5650f9b) of Docker on an Ubuntu 22.04.4 LTS machine with the following specifications:

PC Model: ASUS TUF Gaming fx 505 ge
CPU: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
GPU: NVIDIA GeForce GTX 1050 Ti - 4GB (Driver Version: 535.183.01, CUDA Version: 12.2)
RAM: 16GB

To run the code, the NVIDIA Container Toolkit must be installed following the instructions available here. Also, the steps in the configuration part of the documentation must be followed.

Check the configuration by running cat /etc/docker/daemon.json. The output should be similar to the one below:

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

To test that your setup is working correctly, try to run a sample workload following the official instructions here.

If everything is working correctly, you can proceed to install Isaac Gym by downloading the Isaac Gym Preview 4 package. Unzip the package in a folder of your choice and clone the IsaacGymEnvs repo inside it. Then, edit the docker/Dockerfile file to replace the dependencies installation command with the following:

# dependencies for gym
#
RUN apt-get update \
 && apt-get install -y --no-install-recommends \
 libxcursor-dev \
 libxrandr-dev \
 libxinerama-dev \
 libxi-dev \
 mesa-common-dev \
 zip \
 unzip \
 make \
 gcc-8 \
 g++-8 \
 vulkan-utils \
 mesa-vulkan-drivers \
 pigz \
 git \
 libegl1 \
 git-lfs \
 libsm6 \
 libxext6 \
 libxrender1 

Then, run the following command from the root of the folder to build the Isaac Gym Docker image:

bash docker/build.sh

After the image is built, you have to replace the content of the docker/run.sh file within the Isaac Gym folder with the following code (be careful to replace /path/to/cps_project_folder with the path to the folder containing the code of this repository)

#!/bin/bash
set -e
set -u

export DISPLAY=$DISPLAY
echo "setting display to $DISPLAY"

docker run -dit -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY --gpus=all --name=isaacgym_container --volume=/path/to/cps_project_folder:/opt/isaacgym/cps_project isaacgym 

Then, you can run the Isaac Gym container by running the following command from the root of the Isaac Gym folder:

bash docker/run.sh

Now, you should have a container running with Isaac Gym installed and the code of this repository mounted inside it. If this is the case, you can access it by running the following command:

docker exec -it isaacgym_container bash

As a final step, you need to install the cps_project package and its Python dependencies by running the following command from the root of the mounted folder:

pip install -e .

Finally, you need to install the IsaacGymEnvs package by running the following command inside /opt/isaacgym/IsaacGymEnvs:

pip install -e .

Usage

The code is organized in the following way:

• cps_project package: contains the source code of the project as long as the Isaac Gym tasks
• resources folder: contains the urdf models for the Isaac Gym environments
• scripts folder: contains the scripts to run the training and evaluation of the models
• docs folder: contains the Isaac Gym documentation, some useful resources and the final PowerPoint presentation of the project

To see an example of the environment in which the training is performed, run the following command from the root of the repository (inside the Isaac Gym container):

python scripts/basic_env.py

To run the training for the hovering task, without the obstacles avoidance part, run (always from the root of the repository, inside the Isaac Gym container):

python scripts/ppo_hovering.py

To run the training with cameras and obstacles, run (always from the root of the repository, inside the Isaac Gym container):

python scripts/ppo_cameras.py

By default, wandb is disabled. To enable it, add the flag --wandb and then log in with your account.

Known Issues

• It is common that running the simulation for the first time, the following error is obtained:

  [Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 1373

  Simply re-running the script, the error will not be present anymore.

• A "Not Responding" window for "Isaac Gym" may occasionally pop up, particularly when using wandb. However, the simulation is still running, so there's no need to click "Force Quit." Instead, just select "Wait."

Useful Resources & Links

Here we have collected and categorized various resources that have been useful during the project development

Deep Learning, NN, CNN

Links:

• CNN with MNIST 1
• CNN with MNIST 2
• CNN with MNIST 3
• CNN theory

Books:

• Deep Reinforcement Learning Hands-On: Apply Modern RL Methods, Maxim Lapan
• Foundations of Deep Reinforcement Learning: Theory and Practice in Python

Useful repos

• Aerial Gym Simulator