Improved robot force/tactile sensing capabilities 🤖🖐️

[elle-miller]

The force/tactile sensing modality is becoming increasingly fundamental in robotics research, but Isaac Lab currently offers limited functionality in this area (e.g. compared to MuJoCo Touch Grid).

To keep up to date with what sensors robotics researchers/practitioners use in real life, I am providing 3 feature suggestions that I believe would be highly valuable to the community.

1. Improved API for ContactSensorCfg

Ability to provide force measurements in the frame of the selected body. Currently, the forces are currently being returned in the world frame, which becomes meaningless in many scenarios (i.e. if a manipulation robot rotates). I have had to create a FrameTransformer to track my contact body and then do the quaternion rotation to get the forces in this frame, but it would save us time if this was the default setting.

2. High-resolution tactile sensing

It would be great to have an Isaac Lab equivalent of the MuJoCo Touch Grid! Currently, we are limited to creating physical contact bodies for the sensors, which becomes unfeasible for a higher measurement density, or curved surfaces like fingertips.

It would be fantastic if you could specify a contact body, the number of taxels needed, and the sensor returns the f_normal, f_x_shear, f_y_shear at each taxel.

3. F/T end-effector sensor

Many robots are equipped with a F/T sensor at the wrist, which computes the net forces and torques applied to the end-effector. It is not clear how one could compute this with the current ContactSensorCfg setup.

💗 React to this post if you would also like to see these features! 💗

[RandomOakForest]

Thank you for posting these suggestions. We will tag for review.