High Level Plan for marlowe-sync

[jhbertra]

I'm starting this discussion for us to lay out and agree upon a high-level plan for the new runtime component marlowe-sync (unification for and replacement of marlowe-history and marlowe-discovery).

Problems to Address

1. Following multiple contracts simultaneously is difficult to do in an efficient way (particularly following all contracts) with the current MarloweSync protocol.
2. Internals of marlowe-history are complicated significantly by the requirement to explicitly follow / unfollow contracts, which makes replacing the backend and servicing certain queries hard.
3. Support for additional queries is needed (e.g. Load a contract transaction by ID, get the contract ID associated with a transaction ID, get the tip of a contract)

High Level Plan

1. Build a new Marlowe Runtime component (consisting of 2 sub-components) called marlowe-sync (marlowe-contract-indexer and marlowe-syncd to mirror the design and scalability goals of chain-indexer / chainseekd)
   a. marlowe-contract-indexer connects to chainseekd and transcribes marlowe transactions to a postgres database.
   b. marlowe-syncd binds to a socket and accepts incoming client connections on one of the application protocols.
2. Expose MarlowetSync and MarloweHeaderSync ports from marlowe-syncd
3. Expose a MarloweBulkSync port from marlowe-syncd
   a. Like MarloweSync but it emits the full history of every contract on chain.
4. Write a new query API with support for the following queries
   a. GetContractHeader - returns the header info for a contract
   b. GetContractState - returns the current state of the contract
   c. GetContractForTransaction - returns the contract header for a given Marlowe transaction ID
   d. GetTransaction - returns the transaction information for a given Marlowe transaction ID
5. Rewrite the CLI commands that depend on marlowe-history and marlowe-discovery to depend on marlowe-sync instead
6. Rewrite the REST API endpoints that depend on marlowe-history and marlowe-discovery to depend on marlowe-sync instead.

[bwbush]

Here are the key points about awkward use cases mentioned in the meeting:

1. Query to find the state of a contract at the current tip.
2. Query to list active contracts at the current tip.
3. Query to find the ContractId given a TxId.
4. Bulk follow history of a subset or all contract.
5. Multitenancy for following contracts.
6. Indexing all contracts.
7. General scanning of blocks/transactions/contracts, given a query.
8. Document that Wait only occurs at tip.

[jhbertra]

Note that the multitenancy for following contracts is no longer an issue if we index all contracts and remove the concept of a "follower."