[Question] Problems with Collisions in Pusher-v4

[HernandezEduin]

Question

I'm testing out the RL training with cleanRL, but I noticed in the video provided below that the robotic arm goes through both the table and the object it is supposed to be pushing.

The code below also has a similar behaviour:

import gymnasium as gym
env = gym.make("Pusher-v4", render_mode="human")
observation, info = env.reset(seed=39)
for _ in range(1000):
   action = env.action_space.sample()  # this is where you would insert your policy
   observation, reward, terminated, truncated, info = env.step(action)

   if terminated or truncated:
      observation, info = env.reset()

env.close()

Any idea what is going on or what might be missing?

---

System info

Installed via pip version 0.29.1

Python 3.8.12
Gymnasium 0.29.1
Mujoco 3.1.2

---

rl-video-episode-9000.mp4

[Kallinteris-Andreas]

1. does this also happen with mujoco==2.3.7
2. can you produce the video with a different angle

[HernandezEduin]

1. Yes, same issue with Mujoco==2.3.7.
2. Is there a way to modify the angle with the Gymnasium record_video?

[Kallinteris-Andreas]

Yes, you need to change the azimuth elevation camera properties

import gymnasium as gym
import numpy as np

DEFAULT_CAMERA_CONFIG = {
    "trackbodyid": -1,
    #"distance": 4.0,
    "distance": 3.0,
    "azimuth": 135.0,
    "elevation": -22.5,
}

env = gym.make("Pusher-v5", default_camera_config=DEFAULT_CAMERA_CONFIG, render_mode="human")

[HernandezEduin]

2. The changing of camera option is not present for Gymnasium 0.29.1. If I upgrade to gymnasium==1.0.0a1, the option becomes available along with Pusher-v5, but I'm no longer able to run CleanRL due to compatibilities issues of this version of Gym and Stable-Baselines3.

[Kallinteris-Andreas]

You could try this:

import gymnasium

DEFAULT_CAMERA_CONFIG = {
    "trackbodyid": -1,
    #"distance": 4.0,
    "distance": 3.0,
    "azimuth": 135.0,
    "elevation": -22.5,
}

env = gymnasium.make("Pusher-v4", render_mode="human")
env.unwrapped.mujoco_renderer.default_cam_config = DEFAULT_CAMERA_CONFIG
env.unwrapped.mujoco_renderer._set_cam_config()

[HernandezEduin]

Okay, it seems to be working after I changed the camera angle for some unknown reason.

rl-video-episode-512.mp4

[Kallinteris-Andreas]

Are you using the same policy?
double check: does this also happen for mujoco==3.1.2
double check: does this work without env.unwrapped.mujoco_renderer._set_cam_config()

[HernandezEduin]

Actually, I have to run it without env.unwrapped.mujoco_renderer._set_cam_config(), otherwise it gives me an assertion error on self.viewer is not None. As for whether it works with Mujoco==3.1.2, no. It goes right through the object it seems that the version is causing some errors.

With Mujoco==2.3.7

rl-video-episode-8.mp4

With Mujoco==3.1.2

rl-video-episode-8.mp4

[Kallinteris-Andreas]

also try mujoco==3.1.3 if it does not fix the issue

please find which version of mujoco broke it and report it to the mujoco team (https://github.com/google-deepmind/mujoco) also (tag me in the issue)

Thanks

[pseudo-rnd-thoughts]

@HernandezEduin also, if you have some example actions list for a fixed seed for the MuJoCo team to use, it would be helpful

[HernandezEduin]

Okay, so I've tested it out with various versions of Mujoco and it the results are as follows:

mujoco==2.3.6 - working
mujoco==2.3.7 - working
mujoco==3.0.0 - not working
mujoco==3.0.1 - not working
mujoco==3.1.0 - not working
mujoco==3.1.1 - not working
mujoco==3.1.2 - not working
mujoco==3.1.3 - not working

The collisions stopped working with Ver. 3.0.0. I'll report it soon to the Mujoco team.

[Kallinteris-Andreas]

Continuing from the mujoco issue

Can you render with contact forces? @HernandezEduin

import gymnasium
import mujoco

env = gymnasium.make("Pusher-v4", render_mode="rgb_array")
env.reset()
env.render()
env.unwrapped.mujoco_renderer.viewer.vopt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTFORCE] = True
env.unwrapped.mujoco_renderer.viewer.vopt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTPOINT] = True

...  # run and render your episode here

Thanks!

[pseudo-rnd-thoughts]

@Kallinteris-Andreas given that dm-mujoco don't believe that this is a bug on their end and this is actually just an issue with the environment model.

Does this problem continues in v5, if so we fix it in v5 before the full release of v1.0.0?