[Question] Debugging physics simulation

[vvijay-cov]

Question

I'm trying to simulate a 6DoF robot with a two-finger gripper in Isaac Lab. I've set up a similar environment to the Franka cabinet example, and am trying to move the gripper from the open to the closed position using an action. However, I'm finding that while I can control the robot joints as expected, I can't change the position of the gripper easily. If I set the damping to 0, and the stiffness to a very high value 2e11, I can get the gripper to move, although it still pulsates a lot.

To get to my question, how can I debug this issue further? I've tried stepping through the Isaac lab and Isaac sim code, and it seems like the target positions for the gripper are propagated correctly. Since it seems as though the actual pd controller updates happen inside the Physx simulator, I was wondering if I could get more visibility into it. For example, is there a way to hook up the Physx Visual Debugger to Isaac Sim? Or do you have any other suggestions?

[preist-nvidia]

@vvijay-cov - we have a lot more information, including the internal workings of the drives in PhysX here: https://nvidia-omniverse.github.io/PhysX/physx/5.4.1/docs/Articulations.html#articulation-joint-drives, and in particular, please check out the detailed notes here: https://nvidia-omniverse.github.io/PhysX/physx/5.4.1/_downloads/6acf3afb8f69452757e0e766b5a22978/implicitDrives.pdf

But really, you should need super high gains to get the gripper moving. Do you have any additional constraints that may be active and working against the drives? Given that it moves with high gains this is unlikely, but there's also a max joint velocity and max drive force limit that could be active.

Getting a PVD capture is a good idea, too, you can find instructions here on how to capture from IsaacLab with a couple of lines of Python: https://docs.omniverse.nvidia.com/extensions/latest/ext_omnipvd.html#enabling-omnipvd-recordings-via-python-in-a-composer-session

[varunvijamz]

Thanks, I was able to get some useful information from PVD. The problem was that the gripper was colliding with another body.