feat: private gateway

[fforbeck]

Private Gateway: KMS Integration for Encrypted Content Access

This PR implements the private gateway for handling encrypted content access through a dedicated freeway deployment.

Key Features

Google KMS Integration: Complete RSA key pair setup per space with automatic key generation, integrity validation, and proper space DID mapping.

UCAN Invocation for Encryption Setup: Implementation of space/encryption/setup capability with plan validation and KMS key provisioning for paid users.

UCAN Invocation for Symmetric Key Decryption: Handler for processing symmetric key decryption requests with proper authorization validation and space ownership checks.

Security: Server-side CRC32C validation, standardized error handling, and compliance with Google Cloud KMS security standards.

This enables secure, encrypted content storage and access control for private spaces through a dedicated freeway gateway deployment.

Sequence Diagram: https://gist.github.com/fforbeck/be6486223b387b67f319ba113901da9f

TODO

• Revocation Check will be added in another PR
• Verification if a given space is provided will also be done in a different PR
• Fix broken tests
• Code Audit & Fixes

Code Audit

• Security Posture Summary

  • Critical vulnerabilities: ✅ ALL RESOLVED
  • High-priority vulnerabilities: ✅ 5/7 RESOLVED (71% complete, 1 deferred)
  • Medium-priority vulnerabilities: ✅ 4/5 RESOLVED (80% complete, 1 deferred)
  • Low-priority vulnerabilities: ⚠️ 0/2 IMPLEMENTED (all pending)
  • Remaining blockers: 1 critical security control (revocation checking)

• Report: https://hackmd.io/@fforbeck/H1T7k9YBel

Closes

• New gateway that supports space/encryption/setup invocations project-tracking#497
• KMS Integration project-tracking#498
• Update the Upload and Download flows project-tracking#499

[hannahhoward]

Generally LGTM. Maybe this is a naive question, but at this point, are freeway and the encryption service really the same thing? What I'm asking is the "private gateway" isn't really a gateway no? It's just a key encryption setup + decrypter? And it might be easier to run it out of Amazon where it's right next to the relevant DB tables.

Again, want to keep it moving, but I also want to understand if there is a vision to actually use the "private gateway" as a gateway? (or is that to handle the server side decryption if/when we support it)

[hannahhoward]

are we confident the console.log is going somewhere that's preserved?

[fforbeck]

Good point. I guess I need set up some worker log config to preserve it. I will double-check that. Thanks for flagging.

[fforbeck]

Generally LGTM. Maybe this is a naive question, but at this point, are freeway and the encryption service really the same thing? What I'm asking is the "private gateway" isn't really a gateway no? It's just a key encryption setup + decrypter? And it might be easier to run it out of Amazon where it's right next to the relevant DB tables.

Again, want to keep it moving, but I also want to understand if there is a vision to actually use the "private gateway" as a gateway? (or is that to handle the server side decryption if/when we support it)

Yes. They are different services. This architecture came from initially planning server-side decryption, but I pivoted to client-side key decryption without rethinking the infrastructure to get this rolling faster.

I would suggest a two-phase approach:

1. Validate first: Let's deploy on the current freeway infrastructure at private.storacha.link - quick to market, focus on DMAIL integration
2. Optimize later: Then we migrate to AWS (near DBs) with just a DNS change (DMAIL Integration Phase 2) - users see no difference

WDYT?