Live monitoring of Behavior Tree for many mobile robots

[ljaniec]

Foxglove should support live monitoring of Behavior Trees execution live during navigation of many robots, as a custom panel or extension (similar to Groot and Nav2 stack, but with adequate continuous support for debugging and new features).

Shortly: Since ROS Humble there will be no option to see BT during execution of the BT Navigator from Nav2 stack (ZeroMQ bugs for multirobot systems, it was described in some issue and in "Migration from ROS Galactic to Humble" notes)

• Multi-robot navigation (using multi_tb3_simulation_launch.py) does not work [Groot monitoring] ros-navigation/navigation2#2386
• https://navigation.ros.org/migration/Galactic.html?highlight=multi#bt-navigator-groot-multiple-navigators
• https://navigation.ros.org/migration/Galactic.html?highlight=multi#dropping-support-for-live-groot-monitoring-of-nav2

I see that an option to view in real-time execution of 1+ BT will be an awesome new feature in Foxglove, because since next year and Humble there will be no clear candidate for this.

Best,
Łukasz Janiec

[jtbandes]

Hi, thanks for sharing your feature request!

We are not very familiar with Behavior Trees and Groot. Do you know how popular these tools/libaries are or how many people use them?

It looks like they are currently unmaintained: https://github.com/BehaviorTree/BehaviorTree.CPP / https://github.com/BehaviorTree/Groot

No company, institution or public/private funding is currently supporting the development of BehaviorTree.CPP and Groot. As a consequence, my time to support BehaviorTree.CPP is very limited and I decided that I won't spend any time at all supporting Groot. Pull Requests are welcome and will be reviewed, even if with some delay.

Which features from Groot are most important to you and would be required for a minimum viable solution?

[ljaniec]

Behavior Trees are important tool for robotics, especially ROS 2 and its Navigation2 (Nav2 stack).

Examples of active, maintained packages with use of BT:

• https://github.com/ros-planning/navigation2/tree/main/nav2_behavior_tree
• https://github.com/ros-planning/navigation2/tree/main/nav2_bt_navigator
• https://github.com/Adlink-ROS/BT_ros2

Short intro to BT in Nav2:

• https://navigation.ros.org/concepts/index.html#behavior-trees
• https://navigation.ros.org/tutorials/docs/using_groot.html

There are nav2_bt_navigator in the development too. Groot repository isn't actively maintained, that's why I would like to have other tool for visualization and monitoring of Behavior Trees in ROS2 projects (on simulated and real robots). Foxglove fits this niche perfectly.

Minimum viable solution/features would be:

• display a Behavior Tree like that (picture from Groot with default Nav2 BT):
  
• monitor a Behavior Tree of chosen robot (with option of choice which one):
  
  (picture from https://robohub.org/introduction-to-behavior-trees/)

[jtbandes]

Can you provide or link to some sample bags which contain this data?

[ljaniec]

Yes, of course:

• https://www.behaviortree.dev/xml_format/ (defines XML format of BT)
• https://navigation.ros.org/behavior_trees/index.html and https://navigation.ros.org/behavior_trees/overview/nav2_specific_nodes.html
• https://navigation.ros.org/configuration/packages/configuring-bt-xml.html#configuring-behavior-tree-xml (at the bottom of page) and on GitHub https://github.com/ros-planning/navigation2/tree/main/nav2_behavior_tree and https://github.com/ros-planning/navigation2/tree/main/nav2_bt_navigator/behavior_trees
• https://navigation.ros.org/behavior_trees/overview/detailed_behavior_tree_walkthrough.html (and Navigate To Pose With Replanning and Recovery BT - picture and XML below)

[jtbandes]

Is the BT itself stored in the bag file? And which other message types are used to indicate the current BT state?

[ljaniec]

AFAIK BT isn't stored in a rosbag (it was mentioned there). It seems that BT state could be indicated through currently executed plugin or nodes return values (SUCCESS/RUNNING/FAILURE). Honestly, I don't have any deeper knowledge about it :( I linked this feature request in Nav2 Slack, maybe other more knowledgeable people could help more.

[github-actions[bot]]

This issue has been marked as stale because there has been no activity in the past 6 months. Please add a comment to keep it open.

[adamconkey]

I am very interested in this feature, and I think it would be a great addition to Foxglove Studio in general since BehaviorTree.CPP has become the defacto behavior tree tool for robotics, and Groot is its frontend graphical tool for editing/monitoring the trees. Things seem to be a bit in flux, Groot development has been suspended and only works with BehaviorTree.CPP v3.8 and will soon be replaced by Groot2 that will support the new BehaviorTree.CPP v4. I think the new Groot/BehaviorTree.CPP versions might not ensure back-compatibility, so it'd be wise to target development for the new ones once Groot2 is released.

[facontidavide]

For your information, Nav2 will move to BT.CPP 4.X in Jazzy and this will FINALLY make the old Groot1 obsolete!
Groot2 is ready for prime time and its integration with Foxglove is unlikely.

[tonynajjar]

Groot2 is ready for prime time and its integration with Foxglove is unlikely.

It doesn't have to necessarily be an integration Groot2 (that's even probably hard as Groot2 is in C++ and Foxglove custom panels use TypeScript). Just throwing that out there but maybe Foxglove could sponsor @facontidavide to build a BT panel for Foxglove from scratch? I'm sure it would be a huge addition to all Nav2/BT.CPP users.