IMPORTANT: this is an experimental version. Use at your own risk.
IMPORTANT: This has only been tested for Chia quality check and full proof and nothing else
Passing the Chia quality check within recommended 5s is easy, because this check involves doing ~7 seeks.
The real issue with farming Chia plots in the cloud is doing the full proof, which is ~64 seeks, and this happens after the quality check completes and must be completed and propagated in the Chia distributed network before the 30s timeout.
Those 7+64 seeks, in Chia 1.1.6, happen sequentially, which with variable latency can lead to enormous full proof durations (>1min/proof is not unheard of)
Reads done by the quality check can be reused to do a subsequent full proof, cutting reads from 7+64 down to 64
Chia reads are between 8 and 16KB and they usually happen in two subsequent read() calls.
It's ~50ms faster to get the entire 16KB within one request to Backblaze B2. It is suspected that this gain is so small, because server has some cache of its own (small subsequent reads are faster than random reads). 50ms*64 = 3.2s
Chia 1.1.6 uses chiapos 1.0.2 which runs all read requests in sequence, but community provided PRs which can parallelize those reads. This lets us reduce the full proof cost from total duration of 64 sequential reads over the network down to ~7 (and the first one can be cached, too)
Storage performance may impact your results. Feel free to contact Backblaze B2 support if you notice performance issues.
Having the harvester machine as close as possible (network-wise) to the Backblaze B2 region which hosts our account may help with performance.
NOTE: the following installation instruction is not very secure (it doesn't use checksums to verify integrity of PRs)
In order to make full proof verification viable on Backblaze B2, we need the harvester to have a few things:
- FUSE driver to interface harvester with plots stored in a Backblaze B2 bucket (it also has cache and takes care of the prefetch)
- Modified chiapos to parallelize the full proof reads
- Fast connection to Backblaze B2 - depends on which region your account is in. It's way easier to move harvester to region, than to move region to harvester!
Clone this repository (git clone [email protected]:Backblaze-B2-Samples/b2fs4chia.git), then install it
cd b2fs4chia
pip3 install .
Optionally you can use a venv to install b2fs4chia separately from other python packages.
sudo apt-get install python3-venv
python3 -m venv ~/chia
source ~/chia/bin/activate
install a released version of chia-blockchain
pip install chia-blockchain==1.1.6
if this fails try pip install pip --upgrade first.
First, remove chiapos installed by chia-blockchain:
pip uninstall chiapos
You can make sure that worked by doing:
python -c 'import chiapos'
You should get a ModuleNotFound error.
Next, checkout chiapos repo and install a patched version of it.
git clone https://github.com/Chia-Network/chiapos.git
cd chiapos
git fetch origin pull/239/head:pr239
git checkout 1.0.2
git cherry-pick pr239
python setup.py develop # this step requires cmake > 3.14.0 and python3 headers (`sudo apt-get install python3-dev` on debian-like distros)
You will need a config.yaml file in the folder where you run the fuse driver.
An example config ("config.yaml"):
accountId: <youraccountid>
applicationKey: <yourapplicationid>
bucketId: <yourbucketid>
to get bucketId you can go to you cango to the Backblaze web panel. You can also use b2 get-bucket <bucketname> from B2 command line tool (in case you don't have access to the account via web admin panel, but you just have a key).
mkdir /mnt/b2fs4chia
b2fs4chia /mnt/b2fs4chia --cache_timeout 3600
All commands related to chia have to be run from within the venv created a few steps above. To activate it in a new tab/session:
source ~/chia/bin/activate
chia init # this is only required when running chia for the first time. Output may conatain other commands to perform
chia plots add -d /mnt/b2fs4chia
time chia plots check -n 5 -g "FULL-PLOT-FILE-NAME.plot" # just the name, not the path
note the number of proofs and the time it took to fetch them
MIT license (see LICENSE file)
b2fs4chia is based on b2_fuse by Sondre Engebraaten (MIT license)