universe: recursive tree based universe syncing

[Roasbeef]

This is a brainstorm/tracking issue for a more advanced type of Universe sync. The version I'll likely implement in lightninglabs/tap#7 is a simple variant that scales (bandwidth wise) as O(N*M) where N is the number of assets (each has a distinct Universe root), and M is the final diff set that actually needs to be transmitted. There's another factor of hidden in M, which is the amount of keys sent, but the inclusion proofs are much larger than that.

This "linear" method would look something like:

1. Fetch the known universe roots from the peer.
2. Fetch the set of keys for each universe root you care about, or doesn't match what you have stored on disk.
3. For each set of keys
   a. Perform a local set difference (items in the set that you don't know about)
   b. Query remote party for the inclusion proofs of all those leaves.
4. Query again for the universe root to make sure it matches.

In most cases, it's a noop after step 1, as things fully match.

Instead, we can do something that gives us O(log(N)*M) scaling. For each Universe, we can find the bisection point in a logarithmic amount of steps, then fetch the set of proofs M for them.

This tree-based method looks something like:

1. Fetch the set of known universe roots from the peer (note that for a Multiverse setting, this can run another instance of this protocol in the top-level Universe tree, then recursive down an do another instance).
   a. In addition to the root, this now also returns the left+right sibling hash.
2. If the root matches, exit.
   a. Otherwise, look at the left+right tree. If both of them don't match your local view, the recurse down into both left+right.
   b. Otherwise, recurse down into only one of them (the one that doesn't match).
3. (recursive step, written linearly here for simplicity):
   • At this point we've found a diff, we can either fetch the entire sub-tree (short circuit), or continue to recurse down.
   • If recursing, and you find a sibling that matches (the root didn't match), only

For both of these algorithms, we can also reduce the total bandwidth usage by sending batched proofs. Rather than a proof that reveals only a single leaf, we can create a sparse(r) proof tree, by combining the shared internal nodes of all the proofs.

[ZZiigguurraatt]

The above is a very technical proposal, but I wanted to write some additional context here so that people searching may locate this issue on a variety of other keywords.

Right now, when a universe is synced, all proofs are transferred. This is very inefficient as we have to re-sync the entire past history every time we update. As the number of mints and on chain transfers grows, the number of proofs grows. We need a way to intelligently sync only the changes (delta) since the last sync. I think this issue suggests a way to do that.