[RFW] Cardano DSRs and Encryption

[ProfilaMitchell]

DSR thought process dump

Preface

In short I am trying to prove that a UTxO based/smart contract approach (opposing to metadata based approach or theoretical completely offchain approach) is what we should do, as I intuitively think, but it's not that easy, especially at this stage with what we have (no DIDs, no explicit connections between brand and user, etc). At this stage if we are not looking forward it may seem that simple metadata based approach is good enough

Cardano definitions:

Transaction

an entry written to the blockchain, consists of

• list of input UTxOs and redeemers
• set of output UTxOs and validators
• metadata
• id

UTxO

Stands for Unspent Transaction Output. Transaction output is considered spent if it has been used in a confirmed transaction as an input. (TODO: note about inlineable datums). UTxO consists of

• cardano value
• datum
• validator
  UTxO must contain minimum amount of ADA in its value (~2ADA)

Cardano value

Is a mapping from token id and name to its value. For example, 1ADA + 1ZEKE + some NFT is a cardano value. Simply speaking, a cardano value is a few piles of tokens, where each pile contains only one type of tokens. Tokens by itself are not constrained by anything, they can be transferred freely if they are signed by their owner (a public key or a script). Minting and burning (except for ADA) is regulated with policies, which are special types of onchain scripts.

Datum

A datum is a piece of data, which essentially boils down to raw byte string. Datums are used to record arbitrary data to blockchain. Validators have access to datums that belong to UTxOs, but once UTxO is spent it cannot be used in onchain logic. Modifiable state is modeled by a series of datums where each subsequent datum represents new state. Such states need to be tracked between transactions as transaction contains multiple input and output UTxOs, that is usually done with an NFT (called thread token) that is passed in next UTxO.

Validator

Validator is a function that takes datum, redeemer and some transaction info and returns true or false, meaning whether this UTxO can be spent or not. Transaction info also contains information about all outputs and other inputs, which is useful for composing validators.

Redeemer

Is a parameter that can be specified by the entity that tries to spend a utxo.

Metadata

Arbitrary data bound to a transaction, not that it cannot be used in onchain logic.

Encrypted data onchain

The problem with encrypted data is that is basically unverifiable on chain (by smart contracts).

Asserting DSR flow

What requirements and constraints do we have?

Let define DSR action be user asserting a DSR or brand resolving a DSR

• There must be aggregatable series of actions/interactions (user asserting a DSR and brand confirming it resolved and user confirming that), that has to contain or reference private data in encrypted or hashed form.
• Encrypting as much as possible:
  • Encrypting every action completely. This has an advantage of higher privacy between a user and a brand but the data inside unverifiable, both parties have to rely on some external set of rules to construct a valid series. For example, image a situation when a user asserts a right A, but a brand inadequately responds with resolution for right B. Should we consider this brand action invalid and proceed as it'd never existed? Perhabs, but then the usage of blockchain features is quite limited. And if this sequencing is done via datums (and not metadata) we have a problem where onchain state could be malformed by one of the parties
  • Encrypting only personal data, everything about assertion rights is public. The downside of this is that interaction with the brand is private data itself (probably?). But we can validate that all the interaction is correctly formed.
• (series of actions)/(whole interaction) must follow some rules, how do we deal with their violation? just ignore them or not allow in the first place? We might get fine with just ignoring them where both parties are interested in interaction going well. Where there's a possibility that one party might not want to participate or might want to cheat somehow it could be better approach to strictify the rules, e.g. put them on blockchain

Approach 1: Threads of metadata

When an action happens we write that into transaction metadata, subsequent actions reference this transaction. This is easy approach that can be adopted very easily with existing challenge-fund5 code, modifying it slightly a bit to support other types of data than implemented Right. In every subsequent action except the first one is written to metadata, containing the id of previous transaction. Sort of blockchain over blockchain.

• User-brand have no onchain connection, but have offchain one
• user says forget me, which is stated onchain for the record
• profila on behalf of the user notifies brand offchain
• proof of notification written onchain (what is that? how reliable is that?)
• oneof two:
  • brand doesn't want to go onchain
    • brand resolves and responds
    • prifila writes proof of resolution on behalf of the brand and "closes the case" (if applicable)
  • brand wants to go onchain and join profila platform
    • brand creates profila compliant DID
    • brand resolves, responds and writes proof onchain "closing the case" (if applicable)

=> emerges some notion of "case", let's try to define it:

• case emerges from a connection, explicit or implicit, onchain, or offchain

• case has a lifecycle:

  • user asserts a right
  • one of:
    • brand resolves it
      • case closed
    • brand does not resolve the case
      • data is aggregated by user or profila and sent to offchain court
      • court resolves a case
      • proof of that written onchain
      • case closed

• we can make sure case fully or partially well-formed using onchain validations.

  NOTE: But why do we need that? Sounds cool but why? Let's say we don't have that, which is basicly thread of metadatas approach. All badly-formed data would just be considered incorrect and ignored. The proof of parties of action is on the blockchain. Could it be malformed? Yes. Then what is the point of having it onchain? Timestamps and signatures, sure

Approach 2: UTxO based model of "case"

Keep in mind that in this model everything except PI is public

Once a user wants to assert a DSR he creates a Case UTxO with validator that constraints how that UTxO can be spent (basically regulates who can modify what fields)

Case =
  { user_id :: Pubkey | DID
  , brand_id :: Pubkey | DID | ProfilaDID
  , proof_of_connection :: OnchainConnection | ProofOfOffchainConnection
  , case_id :: ThreadToken?
  , personal_info :: Encrypted info (encrypted how? By brand pubkey? or symmetrically somehow?), Or a link to encrypted info (ipfs?)
  , dsr :: DSR
  , status :: Status
  }

-- proofs here contain personal data, hence must be encrypted/link to encrypted data
-- proofs should probably be optional, need a bit research here (TODO)

Status
  = Initiated -- user initiated DSR assertion
  | Resolved ResolutionProof  -- brand claims resolution
  | Violated ViolationProof -- user claims

-- in case of full resolution (confirmed by a user) user spends the case UTxO and that means it's validated

DSR
  = RightToInformation -- how data is used
  | RightToAccess -- what data do you have?
  | RightToBeForgotten ProofUserIsNotForgotten
  | RightToObjectMarketing (ProofUserWasDirectlyMarketed | ProofOfThatPossibility)
  | RightToOptOut (Something else here)
  | RightToRectification DataToRectify

Note: when case is closed, it's results cannot be used in onchain logic no more. (i.e. smart contracts). We can mint token that would be a proof that case was resolved successfully (or not resolved in time, being a proof of brand's bad behavior), but there must be a clear need for that. How could it be use?

Aterthought: this paragraph looks like trash. There should be some proof of authenticity, otherwise everyone could write Cases to chain, who guarantees they are legitimate? Let's try to define legitimacy of a case:

• A case must be signed by the one who claims it, i.e. user
• An case must be created from an existing onchain connection
• If a connection doesn't exist should we create an unconfirmed brand id/did for consistency? or should a profila did be used as a 3rd party that takes responsibility for brands not being on the platform?
• How do we sign an implicit connection?
• User can close the case at any time
• A brand cannot close the case, it can only claim its resolution
• Case cannot be closed by a brand

Approach 3: Token based (including NFT)

The only use for tokens for DSR assertion is probably proofwise (TODO: research that part). Tokens are good for either valuing things (we don't have values here, neither money nor ratings) or for proofing ownership. There is no explicit notion of ownership here. Subscription is also cannot be a token as it cannot be transferred/sold. User could pack a rights to his data into an NFT, but that is out of scope of subscriptions feature

Approach #4 completely offchain

Two parties can have the similar interaction without blockchain and 3rd party (like Profila) if they follow some set of rules, some protocol. Communication can happen via any private messaging system (e.g. email, chat in messanger, etc.). Advantage of this approach is complete privacy, i.e. even the fact of communication is private. Even with metadata approach we are exposing user-brand communication. We can hide (encrypt/hash) exact DSRs but it'll be visible to an outsider number of interactions, and the parties who interacted. The downside of completely offchain approach is that it's harder to prove things, like a who did what and when. Parties can claim they did send/recieve emails at certain times, but it's hard to reason about that. This part needs more thought process, modeling approach #4 should actually show why do we want blockchain to be used and at what extent.

The argument that blockchain record is "immutable proof" is true in the sense that at some point some information by some party was stated and signed, and it's impossible to go back and change it. But there's no guarantee that that initial information was true in the first place.

Sidenote: that is true with cardano even with UTxOs, on account based chains, like solana (and probably etherium) it's possible to verify what data is being written to an account. So on cardano we have to always filter out UTxOs that are not properly created (invalid). Regarding metadata: solana doesn't have it (implemented via account); etherium -- unknown.

What is the use of "smart contracts" on cardano then? To allow for precise logic of data manipulation, and depending on that data take important decisions: distribute funds (anything valuable actually), state some things that are comcluted from other things. The problem of using smartcontracts here is that all the data input is offchain, and the output is to the offchain world, which makes it a bit difficult to reason why blockchain is needed at all. However in the future, when DIDs are going to be integrated, some balancing with tokens (ZEKE or maybe other tokens), when right ownership of data could be NFTized that all would make much more sense. For now, it's really difficult to reason onchain about completely offchain stuff. However formalizing such reasoning as a smart contract may be useful because that reasoning is checked by everyone, otherwise two parties have to check each other actions and reject actions that are are considered invalid and it's basically sort of blockchain itself. In this case an interaction can be broken by any party, in case of public blockchain it would be impossible (sort of)

Conclusions

Basically choice beetween approach #1 and #2 depends on the set of requirements

Approach #1 pros:

• simple
• there's already POC code that writes data to metadata

Approach #1 cons:

• no data wellformedness guarantee
• possible data aggregation failures due to implicit rules violation
• susceptible to "ddos" attack when a brand is attacked with million cases (is this even possible?)

Approach #2 pros:

• more data well-formed-ness guarantees (how much more? does it worth it?)
• rules governing DSR assertion can be onchain code (smart contract)
• pure distinction between pending cases and closed ones (probably good for performance when indexing blockchain, need more research here)
• there's a "case filing collateral" required by cardano, each case would require ~2ADA to start. Could be good to avoid "ddos" attack on a brand. (After case is closed that collateral is returned to a user)
• possibile integration with other contracts, with DIDs, etc, explicit signatures requirements

Approach #2 cons:

• more complex
• possible pitfalls like cardano transaction size limit, etc.

Conclusion on conclusion

So the choice comes down at what extend do we want to use blockchain capabilities, i.e. onchain verification of data and data manipulations.

Note

When I started my initial research before being tasked with DSR assertion feature (writing to blockchain part) I started imagining a design from DIDs (as basic primitive block), then user-brand connections, and only then I'd start thinking about features on top of connections (like DSR assertions). This is onchain-first backend-first approach. But the situation is a bit different, there's already an established structure of requirements, a plan, and adapting to that is a bit difficult for me. The point of this note, perhabs, is to show a conflict of top-bottom vs bottom-top approach. On the one hand we can design basic blocks first (like DIDs, user-brand connections, etc) and build other more complex concepts on top of this, but it in the end there's a risk to "not to be able to build a building because the fundament is not what we want". On the other hand we can plan overall design first (and specify what do we want to get in the end) and then specify what are the building blocks are, this way there's a risk to "bot be able to build a building because the bricks are wrong size". Typical development process is usually a a balance of top-down and bottom-up approaches.

Notes on encryption and transportation

Encryption

Current implementation in challenge-fund5 stores data in Profila's database. We want to factor out Profila out of the equation to increase trust in the platform.

Option 1 -- symmetric

someKey = genSomeKey()
endata = encrypt_sym(mydata, someKey)
cc = [profila.pk, user.pk, brand.pk].map(pk => encrypt_asym(someKey, pk))
write_to_chain (endata, cc)

Downside: need for secure key exchange scheme
Upside: compacted data => reduced cost

Option 2 -- asymmetric

endata = [profila.pk, user.pk, brand.pk].map(pk => encrypt_asym(mydata, pk))
write_to_chain (endata)

Downsides:

• possibly bigger data size, depending on the number of recipients.
• need for public key exchange
  Upsides:
• no need for secure key exchange scheme
• public key exchange doesn't have to be as secure as in option 1

what if recipient encrypts different data for different recipients? This could be validated given unencrypted data very easily -- just encrypt it with public key and see if it matches. In option 1 that is by design -- there's just one data field.

Could we do it zero knowledge so no data have to be revealed?

Pesistance

If data is written to the blockchain it is going to be persistent there forever. This may cause security concerns. Let's say a brand gets hacked and all its private key is leaked. It would be trivial to decrypt all the personal data ever shared with the brand. If we delegate storage of private information onto other public database, like IPFS it would still be public but could be deleted and if the data was not queried for some time it could be lost from public space. Another benefit is reduced transaction costs. However there's still a security vulnerability if some malevolent party decides to cache all the encrypted data for possible future decryption. This poses the question whether PI, even in encrypted form, should be stored publicly.

In the end we expect two things from PI: to be transported to the party we want, to be persisted for some time, but not forever. We maybe want proofs of data transportation to be publicly persisted forever, but for the data itself it is not required, and maybe even undesireable.

What if the data is going to be exchanged peer to peer? That requires additional transport mechanism. Potentially that could be anything end user trusts. Email, messengers, etc. We cannot support any transport layer, as it would require brands to support million on them, which is just unconvinient. We can support set of them (that could be even voted upon by brands and users), or we can support just one.

Let's say we support some transport method that both user and a brand trusts (in the sense that it's not going to reveal data and not going to store it forever). But how does one party proves to another that it did send a message? When that exchange is done publicly via blockchain (or blockchain + ipfs) that is obvious to the public. In case of private transportation method recieving party can claim it never got the message. But in the end the sending party can just resend the data publicly via blockchain, so there's not really much point for a reciever to claim that the data was never received. Thus if we implement a P2P transport method, there should always be an option of sending data publicly via blockchain.

Delegating storage of PI to IPFS (or other public db) could be done as stage 2 as it gives a bit more privacy and reduces transaction costs.

Can we do it fully zero proof?

Since DIDs are going to be implemented in the end a user wouldn't need to attach his PI onto every transaction, that could be taken from the DIDs. But that's a DID implementation problem, so maybe there's not much point in overthinking it here. Some of the information doesn't have to be zero-proof, like the fact that a user wants a brand to delete his data, or inform him about his data usage. However when a user asserts a data rectification DSR, it contains PI, that ideally we would like to zero-proof. This part needs to be researched more and probably out of the scope for now, especially as its probabilistic nature may require fast offchain transport.

E2E encryption notes

Ideally we would like to have an E2E encryption without profila access to privately shared data between a user and a brand. When a brand is not on the platform, then a user must share his data with profila, simply because there is no other way. In other cases all process could happen without profila involvment, and only when user wants profila's help (when brand doesnt respond or doesn't fullfill DSR request correctly) a user can share his data with profila.

With this scheme a user party would store its private encryption key on its device. This would perfectly work for native client (like for mobile phone), but implementing this in browser flow is tricky. A similar scheme but for signing is implemented in every browser wallet, where a browser extension receives transaction, signes it and sends to blockchain (or it could send it back to the web page code). The tricky part is when we are trying to decrypt a data. Even if we have this decryption extension, how do we present decrypted data to the user? The extension could send it back to the page, but then a user must trust the page to not steal it's data. Or extension could modify the DOM to display unenencrypted data, replacing encrypted data or showing some popup. Another solution is to open a new tab with identical page content but the data would be decrypted there, but this is web 1.0 solution, not suitable for modern web flows.

As a balanced solution I would suggest an extension that would would send unencrypted data back to the page, and if a party wants more security there would be an option of using a native client.

For synchronous encryption the flow would be:

• a page fully or partially prepares and formats data to be encrypted
• the page asks a browser to encrypt it with secret key
• browser extension asks a user to encrypt the data
• user confirms the data to be encrypted, chooses secret key and presses OK
• extension sends encrypted data back to page
• encrypted data is used to build transaction that would be signed with browser wallet

For synchronous decryption the flow would be:

• a page sends encrypted data to the extension
• extension asks user to decrypt data
• user confirms, chooses secret key, presses OK
• extension sends decrypted data back to user

For asynchronous encryption we don't have to use an extension, public keys are public so it could be done by the page. However for more security and privacy we can use flow similar to synchronous encryption

For asynchronous decryption the flow is similar to syncronous decryption'

Credentials (keys) should be per case, not per user-brand connection, and not per user

In the end it actually depends on the choice of what is more or less available of existing infrastructure. For now I did not find any sufficient decrypton browser extension, the closest thing is I found is PGP Anywhere. So we either have to sacrifice web version security and trust by allowing users to store keys on the backend (which doesn't sound good at all) or explore other possibilites, like storing private keys in Web Storage, IndexedDB or Web Crypto.

Metadata DSR approach

Building transaction can be done on the front and on the back. Types here expressed in ML-style syntax, we can do it ts-style as well.

Metadata model

Note: Metadata is encoded in CBOR format, that is more or less compatible with JSON, except the fact that strings and bytestrings must be at most 64 bytes length. We might have to split some data if it's too long.

TODO: add format for tagging profila metadatas (for their filtering during aggregation)

Metadata format requires top level metadata object to be a mapping from numbers (which specify what type is inside) to CBOR value. This way metadata can contain several entries of different types. We have two options here:

• Have one type for all profila metadata entries
• Have several profila types for metadata entries (one for dsrs, one for some other thing)

In the end that is not that important at this point, it would be important if we want to combine many transactins into one. For now I'd suggest two global types for entries:

• encryption public key claim
• dsr assertion actions
  To avoid conflicts with other types we can just choose magic number, or we can derive it somehow. I suggest this function to derive numbers for new entry types:

to_metadata_number_type() {
    (echo -n 'ibase=16; '; (echo $1 | md5sum | cut -c-8 |  tr '[a-f]' '[A-F]')) | bc
}

This way we don't have to come up with magic numbers, we can just derive them from strings

$ to_metadata_number_type "profila-dsr"
2629145947

$ to_metadata_number_type "profila-encryption-pkey-claim"
3936835913

For metadatas which contain any private data we can use this model, which also would allow for offchain PI transporting (in the future):

data MetadataEntry d =
  { public :: d
  , private :: Maybe PrivateData
  }

-- For now
data PrivateData = OnChain
  { encrypted_data :: [(PublicKey, EncryptedData)] -- contains assymetrically encrypted data for a brand and/or for profila
  }

-- In the future:
-- data PrivateData
--   = OnChain  -- private data is stored on the blockchain
--     { encrypted_data :: [(PublicKey, EncryptedData)]
--     }
--   | OffChain
--     { transport :: OffChainTransport
--     , unencrypted_data_hash :: Hash
--     }
--
-- data OffChainTransport
--   = IPFS URI
--   | Email
--   | Others...

DSR Data model

type DSRAssertionEntry = MetadataEntry DSRAction

data DSRPrivateData
  = InitializeDSR
    { brandName :: String
    , brandEmail :: String
    , userFullName :: String
    , userEmail :: String
    , dsr :: DSR
    }
  -- potentially we can add other private data vaiants for other DSR actions

data DSRAction
  = InitializeDSR
    { userPk :: PubKey
    , brandPk :: PubKey
    }
  | ClaimResolved
    { initTxId :: TxId
    }
  | ConfirmResolved
    { initTxId :: TxId
    }
  | ConfirmUnresolved
    { initTxId :: TxId
    }

data DSR = TODO

DSR Backend

Data aggregation must be implemented in form of a function that accepts ReqType and returns ResType. Backend should cache all profila related metadata entries

aggregate :: ReqType -> ResType

data ReqType
  = BrandPubKey PubKey
  | UserPubKey PubKey

type ResType = [(DSRAction, Maybe EncryptedData, Timestamp)]

All returned results must satisfy these conditions, otherwise must be ignored or marked as invalid in returned result:

• InitializeDSR must be signed by userPk
• ClaimResolved must be signed by brandPk
• ConfirmResolved must be signed by userPk
• ConfirmUnresolved must be signed by userPk

Encryption options

Identification of parties (user, brand and profila) should be idally implemented via DIDs. However since that is yet to be implemented and since we have a requirement to assert a right even if a brand is not on the platform (is it even relevant to this point?), we can use public keys. Cardano uses Ed25519 keypairs for signing and verification of transactions. These keypairs are generated from a passphrase of 12 or ?? words (elaborate on this). This is primary identity on cardano. However, that keypair type cannot be used for encryption and decryption directly, for that we have several options:

Derive X25519 key for synchronous encryption

The upside of this approach is that we don't have to perform key exchange between parties, basically "send by wallet address" approach. But this approach allows to generate one shared secret key between two keypairs, and it is constant. Ideally a party should allow access encrypted data access only partialy, like per DSR assertion case. But with this approach all user-brand interaction would be encrypted with a single key.

Use separate designated synchronous keypair for encryption

This approach requires secure key exchange, seems non applicable to our use case.

Use separate designated asynchronous keypair for encryption

Pros:

• we can choose any encryption algorithm
• allows for key updates
• only data recipient would be able to decrypt data
• no need for key exchange during DSR process
  Cons:
• requires public key claims scheme
• requires public keys storage/cache

Async encryption data model

A user claims his encryption public key by sending a transaction with this metadata:

data EncryptionPublicKeyClaimMetadata = EncryptionPublicKeyClaimMetadata
  { signingPublicKey :: CardanoSigningPublicKey
  , encryptionPublicKey :: EncryptionPublicKey
  }

TODO: figure out the best algorithm for encryption, probably from Web Crypto supported list.

Encryption backend

A party in order to participate in the profila platform must claim its encryption public key, submitting a transactioon with EncryptionPublicKeyClaimMetadata structure. This transaction must be signed with signingPublicKey to be considered valid. This backend would be a public cache, mapping signing keys (cardanot wallets) to encryption keys.

For a start we can use centralized storage DB (even redis would be fine) and do not submit any pubkey claims to the blockchain.

Building transactions could be done on the backend or on the front. (TODO: research)

Future possibilites:

• updating public encryption keys
• invalidating public encryption keys
• adding public encryption keys

Frontend lib

There should be a front end lib that handles encryption/decryption of the data, it should wrap backend and on top of that do private data aggregation (if its private, like email. IPFS data we can return with backend. TODO: research this part).

getBrandData :: (PubKey, PrivateKey) -> [(DSRAction, Maybe DSRPrivateData, Timestamp)]

For the user function we can use pure aggregate from the backend, as decryption is not needed, we have the data in the blockchain. For profila side data aggregation we can use getBrandData. TODO: research profila role here

Infrastructure

Flows

Server-side encryption

In ideal decentralized web3 world that is not something that should be done. However there's an apparent requirement for that:
Consider a case where there's a big brand or Profila organization itself, that have a lot of request, and those requests are processed by multiple people. It would be unwise to give those people private encryption and signing keys. Changing/updating/rotating keys should be a thing, but it has a lot of cost as well. A bit more simpler solution is local centralization -- profila or a big brand can have a server where private keys would be stored, and only authorized admins/operators could use it to sign and decrypt transactions and data. This part is yet to be formalized and it would also require a backend sending tx flow (we could probably use ogmios for that).

Approximate tasks

• e2e encryption (depends on lib, encryption backend, backend DSR):

  • frontend:
    • cardano-wallet integration (signing, getting keys, etc)
    • local crypto storage integration

• server encryption (depends on lib, encryption backend, backend DSR):

  • backend:
    • encryption private keys storage
    • backend to encrypt and decrypt
  • frontend:
    • pages to manage decryption for brand super admin private keys

• encryption backend (depends on lib):

  • oura filter for on-chain key claims
  • encryption pub key db/cache

• backend DSR (depends on lib):

  • oura filter for on-chain DSR transactions
  • DSR storage/cache

• lib:

  • type model for encryption stuff
  • type model for DSRs
  • metadata and tx building
  • encryption and decryption functions