How to implement an InteractiveMarker plugin?

[schornakj]

I saw that the Interactive Marker is already included in the wiki's list of features needed for Rviz2 parity. There isn't very much discussion about the architectural and technical route forward, other than this note under the "Tools" section:

"A lot of these tools require interaction with the scene which is not available in the raw Scene3D plugin provided by ign-gui. We can either integrate them into the Scene3D plugin or create a custom scene plugin like GzScene3D."

I'm curious about what is missing from the Scene3D plugin that prevents interaction with the scene, and about what kinds of interactions are not possible within the current implementation.

I've done a little experimentation while trying to create a plugin that allows me to click on a 3D shape within the scene but it feels like I've hit a bit of a wall, so I'm wondering if my own challenges are related to the limitations alluded to on the wiki.

[chapulina]

I believe the main missing piece is that the scene needs to broadcast events like clicks and hover. We've been doing this on ign-gazebo's Scene3D, but haven't ported it to ign-gui's Scene3D. I just ticketed gazebosim/gz-gui#209 for this.