How to manage a factor graph that changes (grows/shrinks) over time (e.g. sliding window)?

[scott-robotics]

Hi!

There are a number of scenarios that may require a factor graph to change. A common case would be sliding-window BA for visual-odometry. I may want to prune nodes/factors from the graph based on time/distance/etc and I wondered is there was a recommended way to do this? I have seen others maintain the set of measurements in the window in whatever desired dataypes, the rebuild and reoptimize the graph each time. This is the way I have been doing it too.

The main challenge I'm trying to overcome is that when I want to prune a node/symbol, I need to know all the connected factors to remove, then ensure the factor removal doesn't leave any dangling symbols. So it becomes a bit of a recursive problem. Any insight would be appreciated!