Merge pull request #7713 from freedomofpress/api-v2-docs

docs: state machine and commentary for the v2 Journalist API

Author: vickiniu

API2.md

@@ -1,24 +1,56 @@
 # Journalist API v2
 
-This package (in `securedrop/journalist_app/api2`) implements and documents the synchronization strategy for the v2
-Journalist API.
-
-| File                                  | Contents                             |
-| ------------------------------------- | ------------------------------------ |
-| `README.md`                           | Specification                        |
-| `__init__.py`                         | Server implementation                |
-| `../../tests/test_journalist_api2.py` | Test suite for server implementation |
+The `securedrop.journalist_app.api2` package implements the synchronization
+strategy for the v2 Journalist API.
+
+| File/module                             | Contents                                                                                                       |
+| --------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
+| `API2.md` (you are here)                | Specification                                                                                                  |
+| `securedrop.journalist_app.api2`        | Flask blueprint for `/api/v2/`                                                                                 |
+| `securedrop.journalist_app.api2.events` | Event-handling framework                                                                                       |
+| `securedrop.journalist_app.api2.shared` | Helper functions factored out of and still shared with the v1 Journalist API (`securedrop.journalist_app.api`) |
+| `securedrop.journalist_app.api2.types`  | Types                                                                                                          |
+| `securedrop.tests.test_journalist_api2` | Test suite for server implementation                                                                           |
 
 A client-side implementation should be able to interact with the endpoints
-implemented in `__init__.py` according to this specification.
+implemented in `securedrop.journalist_app.api2` according to this specification.
+
+## Audience
+
+This API is intended for use by the [SecureDrop journalist app][app], and this
+documentation is intended to support its development. We make no guarantees
+about support, compatibility, or documentation for other purposes.
+
+[app]: https://github.com/freedomofpress/securedrop-client/tree/main/app
+
+## Goals and properties
+
+Although the SecureDrop Server remains the source of truth for its clients, the
+v2 Journalist API borrows ideas from distributed systems and content-addressable
+storage.
+
+1. Support the Journalist API's "occasionally connected" clients: actions should
+   be possible while in offline mode, responsive even over flaky Tor connections,
+   etc.
+
+2. Provide a single write-read loop in every synchronization round trip, at an
+   interval of the client's choosing.
+
+3. Hash a canonical representation of each record (source, item, etc.) to
+   version it deterministically.
+
+4. Hash a canonical representation of an endpoint's entire state (all sources,
+   all items, etc.) to version it deterministically.
 
 ## Overview
 
 The request/response schemas referred to in these sequence diagrams are defined
-as mypy types in `__init__.py`.
+as mypy types in `securedrop.journalist_app.api2.types`.
 
 ### Initial synchronization
 
+**Figure 1.**
+
 ```mermaid
 sequenceDiagram
 participant Client
@@ -38,6 +70,8 @@ Server ->> Client: MetadataResponse
 
 ### Incremental synchronization
 
+**Figure 2.**
+
 ```mermaid
 sequenceDiagram
 participant Client
@@ -64,6 +98,8 @@ end
 
 ### Batched events from client
 
+**Figure 3.**
+
 ```mermaid
 sequenceDiagram
 participant Client
@@ -87,10 +123,88 @@ Note over Client: Global version uvwxyz
 end
 ```
 
+#### State machine
+
+Events in a given `BatchRequest` are handled in [snowflake-ID](#snowflake-ids)
+order. Each event is handled according to the following state machine:
+
+**Figure 4.**
+
+```mermaid
+stateDiagram-v2
+direction TB
+
+[*] --> CacheLookup : process(event)
+CacheLookup: status = redis.get(event.id)
+
+CacheLookup --> IdempotentBranch : status in {102 Processing, 200 OK}
+CacheLookup --> StartBranch : status == None
+
+state "Enforce idempotency" as IdempotentBranch {
+    AlreadyReported : 208 AlreadyReported
+    AlreadyReported --> [*] : return AlreadyReported
+}
+
+state "Start processing" as StartBranch {
+    [*] --> Processing : redis.set(event.id, Processing, ttl)
+    Processing : 102 Processing
+}
+
+Processing --> Handler
+state "handle_<event.type>()" as Handler {
+    [*] --> [*]
+}
+
+Handler --> OK
+state "Cache and report success" as SuccessBranch {
+    OK : 200 OK
+    OK --> UpdateCache
+
+    UpdateCache : redis.set(event.id, OK, ttl)
+    UpdateCache --> [*] : return (OK, delta)
+}
+
+Handler --> BadRequest
+Handler --> NotFound
+Handler --> Conflict
+Handler --> Gone
+Handler --> NotImplemented
+state "Report error" as ErrorBranch {
+    BadRequest : 400 BadRequest
+    NotFound : 404 NotFound
+    Conflict : 409 Conflict
+    Gone : 410 Gone
+    NotImplemented : 501 NotImplemented
+
+    BadRequest --> ClearCache
+    NotFound --> ClearCache
+    Conflict --> ClearCache
+    Gone --> ClearCache
+    NotImplemented --> ClearCache
+
+    ClearCache : redis.delete(event.id)
+    ClearCache --> [*] : return error
+}
+```
+
+**Notes:**
+
+1. A client that submits a successful event $E$ will receive HTTP `200 OK` for
+   $E$ and SHOULD apply the event locally as confirmed based on the returned data
+   (`sources`, `items`, etc.).
+
+2. A client that subsequently resubmits $E$ will receive only a cached HTTP `208
+Already Reported` and SHOULD apply the event locally as confirmed. The server
+   will not return data in this case, but the client SHOULD already know the
+   results of the operation once confirmed.
+
+3. A client that submits a failed event $E'$ will receive an individual error
+   code for $E'$. The client MAY resubmit $E'$ immediately, since idempotence is
+   not enforced for error states.
+
 #### Consistency
 
-This diagram implies single-round-trip consistency. To make that expectation
-explicit:
+Figure 3 above depicts single-round-trip consistency. That is:
 
 1. If the server $S$ currently has exactly one active client $C$; and
 
@@ -101,6 +215,10 @@ E_n\}$; and
 
 4. $C$'s index SHOULD match $S$'s index without a subsequent synchronization.
 
+This property does not hold for resubmitted events (returning HTTP `208 Already
+Reported)`. A subsequent synchronization MAY be necessary for the client to
+"catch up" to the effects of accepted events.
+
 #### Snowflake IDs
 
 The `Event.id` field is a "snowflake ID", which a client can generate using a