-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
14 changed files
with
1,263 additions
and
71 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
604 changes: 604 additions & 0 deletions
604
...hain-sharded-storage-web2-costs-and-web3-security-with-shamir-secret-sharing.md
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| --- | ||
| title: Fast Fourier Inspired Folding for Sangria | ||
| date: 2023-04-03 | ||
| --- | ||
|
|
||
| This is a crosspost with [zkresear.ch/t/fast-fourier-inspired-sangria](https://zkresear.ch/t/fast-fourier-inspired-sangria). | ||
|
|
||
| ## Introduction | ||
|
|
||
| [Sangria](https://geometry.xyz/notebook/sangria-a-folding-scheme-for-plonk) is the folding protocol for Plonk prover. In the original model, the prover works iteratively and merges a new execution trace with an execution trace accumulator. | ||
|
|
||
| Here we will show, how to build an optimized folding process, requiring only 2 or 1 scalar multiplications per folding on the verifier side. | ||
|
|
||
| ## Original Protocol | ||
|
|
||
| Accumulated instance and witness: | ||
|
|
||
| $U'_n := (\mathbf{X}'_n, u'_n, \overline{W}'_n, \overline{E}'_n),$ | ||
| $W'_n := (\mathbf{W}'_n, \mathbf{e}'_n, r'_{Wn}, r'_{En})$ | ||
|
|
||
| Iteration instance and witness: | ||
|
|
||
| $U_n := (\mathbf{X}_n, u_n, \overline{W}_n, \overline{E}_n),$ | ||
| $W_n := (\mathbf{W}_n, \mathbf{e}, r_{Wn}, r_{En})$ | ||
|
|
||
| Where $\overline{W}=\text{Com}(\text{pp}_W, \mathbf{W}, r_W),\ \overline{E} = \text{Com}(\text{pp}_W, \mathbf{e}, r_E)$. | ||
|
|
||
| We use a relaxed Plonk gate equation: | ||
| $C(\mathbf{a}, {\mathbf{b}}, {\mathbf{c}}, u, {\mathbf{e}})={\mathbf{a}} {\mathbf{b}} {\mathbf{q_M}} + {\mathbf{q_C}} {u}^{2} + {\left({\mathbf{a}} {\mathbf{q_L}} + {\mathbf{c}} {\mathbf{q_O}} + {\mathbf{b}} {\mathbf{q_R}}\right)} {u} + {\mathbf{e}}$ | ||
|
|
||
| 1. Prover send to Verifier $\overline{T}_n = \text{Com}(\text{pp}_W, \mathbf{t}_n, r_{Tn})$, | ||
| ## Introduction | ||
|
|
||
| [Sangria](https://geometry.xyz/notebook/sangria-a-folding-scheme-for-plonk) is the folding protocol for Plonk prover. In the original model, the prover works iteratively and merges a new execution trace with an execution trace accumulator. | ||
|
|
||
| Here we will show, how to build an optimized folding process, requiring only 2 or 1 scalar multiplications per folding on the verifier side. | ||
|
|
||
| ## Original Protocol | ||
|
|
||
| Accumulated instance and witness: | ||
|
|
||
| $U'_n := (\mathbf{X}'_n, u'_n, \overline{W}'_n, \overline{E}'_n),$ | ||
| $W'_n := (\mathbf{W}'_n, \mathbf{e}'_n, r'_{Wn}, r'_{En})$ | ||
|
|
||
| Iteration instance and witness: | ||
|
|
||
| $U_n := (\mathbf{X}_n, u_n, \overline{W}_n, \overline{E}_n),$ | ||
| $W_n := (\mathbf{W}_n, \mathbf{e}, r_{Wn}, r_{En})$ | ||
|
|
||
| Where $\overline{W}=\text{Com}(\text{pp}_W, \mathbf{W}, r_W),\ \overline{E} = \text{Com}(\text{pp}_W, \mathbf{e}, r_E)$. | ||
|
|
||
| We use a relaxed Plonk gate equation: | ||
| $C(\mathbf{a}, {\mathbf{b}}, {\mathbf{c}}, u, {\mathbf{e}})={\mathbf{a}} {\mathbf{b}} {\mathbf{q_M}} + {\mathbf{q_C}} {u}^{2} + {\left({\mathbf{a}} {\mathbf{q_L}} + {\mathbf{c}} {\mathbf{q_O}} + {\mathbf{b}} {\mathbf{q_R}}\right)} {u} + {\mathbf{e}}$ | ||
|
|
||
| 1. Prover send to Verifier $\overline{T}_n = \text{Com}(\text{pp}_W, \mathbf{t}_n, r_{Tn})$, | ||
| where $t_n=2 \, {\mathbf{q_C}} {u'_n} {u_n} + {\left({\mathbf{a}_n} {\mathbf{b}'_n} + {\mathbf{a}'_n} {\mathbf{b}_n}\right)} {\mathbf{q_M}} + {\left({\mathbf{a}_n} {\mathbf{q_L}} + {\mathbf{c}_n} {\mathbf{q_O}} + {\mathbf{b}_n} {\mathbf{q_R}}\right)} {u'_n} + {\left({\mathbf{a}'_n} {\mathbf{q_L}} + {\mathbf{c}'_n} {\mathbf{q_O}} + {\mathbf{b}'_n} {\mathbf{q_R}}\right)} {u_n}$ | ||
| where $t_n=2 \, {\mathbf{q_C}} {u'_n} {u_n} + {\left({\mathbf{a}_n} {\mathbf{b}'_n} + {\mathbf{a}'_n} {\mathbf{b}_n}\right)} {\mathbf{q_M}} + {\left({\mathbf{a}_n} {\mathbf{q_L}} + {\mathbf{c}_n} {\mathbf{q_O}} + {\mathbf{b}_n} {\mathbf{q_R}}\right)} {u'_n} +\\ {\left({\mathbf{a}'_n} {\mathbf{q_L}} + {\mathbf{c}'_n} {\mathbf{q_O}} + {\mathbf{b}'_n} {\mathbf{q_R}}\right)} {u_n}$ | ||
| 2. Verifier sends to prover random $r$ | ||
| 3. Prover and Verifier output the folded instance | ||
| $U'_{n+1}=(\mathbf{X}'_{n+1}, u'_{n+1}, \overline{W}'_{n+1}, \overline{E}'_{n+1}),$ | ||
| where | ||
| $\mathbf{X}'_{n+1} = \mathbf{X}'_n + r \mathbf{X}_n,$ | ||
| $u'_{n+1} = u'_n + r u_n,$ | ||
| $\overline{W}'_{n+1} = \overline{W}'_n + r \overline{W}_n,$ | ||
| $\overline{E}'_{n+1} = \overline{E}'_n + r^2 \overline{E}_n - r \overline{T}_n.$ | ||
| 4. Prover output the folded witness | ||
| $W'_{n+1} = (\mathbf{W}'_{n+1}, \mathbf{e}'_{n+1}, r'_{W\ n+1}, r'_{E\ n+1}),$ | ||
| where | ||
| $\mathbf{W}'_{n+1} = \mathbf{W}'_n + r \mathbf{W}_n,$ | ||
| $\mathbf{e}'_{n+1} = \mathbf{e}'_n + r^2 \mathbf{e}_n - r \mathbf{t}_n,$ | ||
| $r'_{W\ n+1} = r'_{W\ n} + r r_{Wn},$ | ||
| $r'_{E\ n+1} = r'_{E\ n} + r^2 r_{En} - r r_{Tn}.$ | ||
|
|
||
| We can check, that $C(\mathbf{a}'_{n+1}, \mathbf{b}'_{n+1}, \mathbf{c}'_{n+1}, u'_{n+1}, \mathbf{e}'_{n+1}) = C(\mathbf{a}'_n, \mathbf{b}'_n, \mathbf{c}'_n, u'_n, \mathbf{e}'_n) + r^2 C(\mathbf{a}_n, \mathbf{b}_n, \mathbf{c}_n, u_n, \mathbf{e}_n)$. | ||
|
|
||
| ## Fast-Fourier Inspired Approach | ||
|
|
||
| We see, that most operations on the verifier side are linear. So, we can use the approach from [GW21](https://eprint.iacr.org/2021/1167.pdf). | ||
|
|
||
| Let's define the following functions: | ||
|
|
||
| $f_L(X) = a(X^4) + X b(X^4) + X^2 c(X^4) + X^3 e(X^4),$ | ||
| $f_R(X) = a(X^4) + X b(X^4) + X^2 c(X^4) - X^3 t(X^4),$ | ||
| $\epsilon_R(X) = X^3 e(X^4)$ | ||
|
|
||
| where $a, b, c, e, t$ are polynomials corresponding to vectors $\mathbf{a}, \mathbf{b}, \mathbf{c}, \mathbf{e}, \mathbf{t}$. It is important, that the field has a multiplicative subgroup of order $4$. If we need more columns, we can use the same approach with bigger subgroups. | ||
|
|
||
| Then we can rewrite the witness part of the folding procedure as follows: | ||
|
|
||
| 1. Prover computes $\mathbf{t}$ and sends to verifier $[f_{R\ n}]$, $[\epsilon_{R\ n}]$ | ||
| 2. Verifier sends to prover random $r$ | ||
| 3. Prover and Verifier output the folded instance | ||
|
|
||
| $[f'_{L\ n+1}] = [f'_{L\ n}] + r [f_{R\ n}] + r^2 [\epsilon_{R\ n}]$ | ||
|
|
||
| 4. Prover output the folded witness | ||
|
|
||
| $f'_{L\ n+1} = f'_{L\ n} + r f_{R\ n} + r^2 \epsilon_{R\ n},$ | ||
|
|
||
|
|
||
| For final check we should make openings of $f_L$ at points $x, x \sqrt{-1}, -x, -x \sqrt{-1}$, where $x$ is random, and recover $a(x^4), b(x^4), c(x^4), e(x^4)$. | ||
|
|
||
| It is important to note that the folding process complexity is still linear. We don't need an explicit representation of f(x) in the prover-side folding process: | ||
|
|
||
| $[f_L(x)] = [\sum_{i=0}^{n-1} (a_i + b_i x \lambda_i(x^4) + c_i x^2 \lambda_i(x^4) + e_i x^3 \lambda_i(x^4))] =\\ | ||
| \sum_{i=0}^{n-1} ([a_i] + b_i [x \lambda_i(x^4)] + c_i [x^2 \lambda_i(x^4)] + e_i [x^3 \lambda_i(x^4)]).$ | ||
|
|
||
|
|
||
| The proposed method provides only 2 scalar multiplications per folding instead of 5 or more. And it requires 4 times bigger CRS. | ||
|
|
||
| UPD: | ||
| In the case of IVC, when the 2nd instance is original Plonk, $\epsilon_R(X)=0$ and we need only one scalar multiplication per folding. | ||
|
|
87 changes: 87 additions & 0 deletions
87
research/minimal-fully-recursive-zkda-rollup-with-sharded-storage.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| --- | ||
| title: Minimal fully recursive zkDA rollup with sharded storage | ||
| date: 2024-03-16 | ||
| --- | ||
|
|
||
| ## Current zk rollup state | ||
|
|
||
| zkRollups scale execution efficiently, but publish all blocks at L1. This is not scalable for storage and forbids recursive rollups: if we deploy a rollup on a rollup, we need to publish all the blocks of the inner rollup on the outer rollup, and the outer rollup will publish all its blocks on L1. | ||
|
|
||
|  | ||
|
|
||
| There were some attempts to solve this problem, like validiums, but they are weak on both decentralization and security (2 of 3 in Vitalik's trilemma). | ||
|
|
||
|
|
||
| ## Existing improvements in unlocking data availability and decentralized storage | ||
|
|
||
| ### Chia | ||
|
|
||
| Chia introduced a novel consensus algorithm called Proof of Space and Time (PoST), which provides a more decentralized and energy-efficient alternative to Proof of Work (PoW): Proof of Space-Time (PoST). PoST is a consensus algorithm that uses storage space as a resource to secure the network. | ||
| The current capacity of Chia Network is 33 EiB. | ||
|
|
||
|
|
||
| ### EthStorage | ||
|
|
||
| Ethstorage is replication-based DA and storage, managed by a smart contract. | ||
|
|
||
|
|
||
| ## Our results | ||
|
|
||
| In our [research draft](https://ethresear.ch/t/blockchain-sharded-storage-web2-costs-and-web3-security-with-shamir-secret-sharing/18881) we propose a solution for storage and data availability, friendly to zk rollups and unlocking new scalability opportunities. | ||
|
|
||
| ### Sharding instead of replication | ||
|
|
||
| It is proposed to use $k$ of $n$ threshold data representation. So, any $k$ numbers from the source file are transformed into $n$ numbers. And any $k$ of these $n$ numbers can restore the source $k$ numbers. This is called Shamir's Secret Sharing. | ||
|
|
||
| This approach allows us to utilize storage 10-20 times more efficiently than the replication-based approach, according to our modeling. | ||
|
|
||
| Also, it gives us better protection from physical-level attacks, like target node destruction. | ||
|
|
||
| ### Unlimited horizontal scalability | ||
|
|
||
| We propose to use a 2-level nested rollup structure (below we will describe, why it is possible). The top-level rollup manages participants of low-level rollups and mixes them to prevent the accumulation of malicious participants in one low-level rollup. Low-level rollups manages the data, stored in the nodes. | ||
|
|
||
| ### Polynomial commitments everywhere | ||
|
|
||
| We propose to use Merkle trees on the top level of database. However, the minimal structure is a polynomial commitment to a cluster of data. So, it is very friendly to rollups, because we can use the same polynomial commitment to represent the rollup's block. | ||
|
|
||
| Also, out of the box we have data availability oracle (just provide random polynomial lookup on the commitment) and all linear algebra we needed for sharding. | ||
|
|
||
| ### Data mining | ||
|
|
||
| Nodes can use the data for mining, like in Chia. And the result of mining is zero-knowledge proof of data availability. | ||
|
|
||
| The complexity of storage is leveled, so it is the same complexity to store random data or zeros. | ||
|
|
||
| Nodes can join to network with trustless zk proof of their capacity. | ||
|
|
||
| ## Bring it all together | ||
|
|
||
| ZK Rollups usually publish on-chain proof of execution and data of the block. | ||
| But our data availability and proof of storage are zk. So, we can merge it all together and publish the proof of execution and data availability and storage in one single ZK proof. | ||
|
|
||
| It unlocks the deployment of rollups on rollups, and the rollups on rollups on rollups, and so on. And way to transform Web2 into Web3. | ||
|
|
||
| Also, we can prevent the bloating of the blockchain: if we publish the snapshot state of the rollup, previous history could be removed. | ||
|
|
||
|  | ||
|
|
||
|
|
||
| ## Some economics | ||
|
|
||
| On 1st Jan 2024 cost of storage, 1GiB was: | ||
|
|
||
| * Ethereum $1.8M | ||
| * EthStorage $10k | ||
| * Celestia $300 | ||
| * Near $10 | ||
|
|
||
| Based on [Hetzner sx294](https://www.hetzner.com/dedicated-rootserver/sx294/) with 8 blowup factor (what we need for >100 bits of security), the annual cost of storage 1GB is $0.15 usd. | ||
|
|
||
| The cost will be lower on specialized rigs. | ||
|
|
||
| ## Call for discussion and feedback | ||
|
|
||
| We believe our proposed solution has the potential to significantly improve the scalability and efficiency of zk rollups and upgrade Web2 to Web3. However, we acknowledge that this is still a research draft and there may be challenges or considerations we haven't fully addressed. | ||
|
|
||
| We welcome discussion, feedback, and constructive criticism from the community. If you have insights, ideas, or see potential issues with our approach, please share them. |
Oops, something went wrong.