Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DataCloneError - requirements on nightly + build_std=std #1989

Open
dannywillems opened this issue Jan 28, 2025 · 0 comments
Open

DataCloneError - requirements on nightly + build_std=std #1989

dannywillems opened this issue Jan 28, 2025 · 0 comments

Comments

@dannywillems
Copy link
Member

I leave this issue open in case someone else encounter the same, or try to get rid of nightly.

While working on MinaProtocol/mina#16464, I got rid of nightly to build o1js, to use a single Rust version, and use cargo vendor. A direct consequence is to get rid of the unstable flags -Z build_std=std,abort.
However, this gives the following error:

DOMException [DataCloneError]: #<Memory> could not be cloned.
    at new DOMException (node:internal/per_context/domexception:53:5)
    at new Worker (node:internal/worker:266:17)
    at <anonymous> (o1js/o1js/src/bindings/js/node/node-backend.js:56:20)
    at Function.from (<anonymous>)
    at startWorkers (o1js/o1js/src/bindings/js/node/node-backend.js:55:11)
    at module.exports.__wbg_startWorkers_f430e3766320935f (o1js/o1js/dist/node/bindings/compiled/_node_bindings/plonk_wasm.cjs:9782:17)
    at wasm://wasm/00e87c76:wasm-function[2310]:0x3607fd
    at module.exports.initThreadPool (o1js/o1js/dist/node/bindings/compiled/_node_bindings/plonk_wasm.cjs:1561:22)
    at initThreadPool (o1js/o1js/src/bindings/js/node/node-backend.js:34:14)
    at withThreadPool (o1js/o1js/src/lib/proof-system/workers.ts:41:27)
    at compileProgram (o1js/o1js/src/lib/proof-system/zkprogram.ts:834:7)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Function.compile (o1js/o1js/src/lib/mina/zkapp.ts:655:48)
    at async file:///home/runner/work/o1js/o1js/src/examples/zkapps/reducer/run-live.tmp.js:105:1

See https://github.com/o1-labs/o1js/actions/runs/12995408239/job/36242047796?pr=1986 for instance.
dannywillems added a commit to MinaProtocol/mina that referenced this issue Jan 28, 2025
@dannywillems
kimchi-bindings/js: back on nightly and use 2023-09-18
207c531 
nightly has been initially removed as it seemed to make vendoring all the
dependencies more complicated (which seems to not be the case though).
However, removing nightly introduces issues like "DataCloneError", see
o1-labs/o1js#1989, which makes the reliance on
nightly inevitable.
It seems that --offline works out of the box.

Another note is that cargo vendor has been introduced to provide a
consistent build environment for all platforms and all users, including
the CI. The lockfile Cargo.lock is meant to provide a locked environment
with all the dependencies, including the transitive ones. However, it is
only used by binaries. In our build process, we do build a static
library (libwires_15_stubs.a), not a binary. And Rust doesn't use the
lockfile in this case.
Another solution than `cargo vendor` would have been to strictly enforce
the version in the top-level Cargo.toml, but it implies to add all the
transitive dependencies (as otherwise, when a new transitive dependency
is released, cargo will fetch it, making the environment inconsistent).
This solution is not viable as the purpose of Cargo.toml is to enforce
versions on direct dependencies only, and leave the cargo solver doing
its job for the transitive dependencies.
For these reasons, the solution of vendoring the dependencies to build
the static library has been selected.
@dannywillems dannywillems mentioned this issue Jan 28, 2025
Draft
dannywillems added a commit to MinaProtocol/mina that referenced this issue Jan 28, 2025
@dannywillems
kimchi-bindings/js: back on nightly and use 2023-09-18
6e8c1b0 
nightly has been initially removed as it seemed to make vendoring all the
dependencies more complicated (which seems to not be the case though).
However, removing nightly introduces issues like "DataCloneError", see
o1-labs/o1js#1989, which makes the reliance on
nightly inevitable.
It seems that --offline works out of the box.

Another note is that cargo vendor has been introduced to provide a
consistent build environment for all platforms and all users, including
the CI. The lockfile Cargo.lock is meant to provide a locked environment
with all the dependencies, including the transitive ones. However, it is
only used by binaries. In our build process, we do build a static
library (libwires_15_stubs.a), not a binary. And Rust doesn't use the
lockfile in this case.
Another solution than `cargo vendor` would have been to strictly enforce
the version in the top-level Cargo.toml, but it implies to add all the
transitive dependencies (as otherwise, when a new transitive dependency
is released, cargo will fetch it, making the environment inconsistent).
This solution is not viable as the purpose of Cargo.toml is to enforce
versions on direct dependencies only, and leave the cargo solver doing
its job for the transitive dependencies.
For these reasons, the solution of vendoring the dependencies to build
the static library has been selected.
dannywillems added a commit to MinaProtocol/mina that referenced this issue Jan 28, 2025
@dannywillems
kimchi-bindings/js: back on nightly and use 2023-09-18
3d2484c 
nightly has been initially removed as it seemed to make vendoring all the
dependencies more complicated (which seems to not be the case though).
However, removing nightly introduces issues like "DataCloneError", see
o1-labs/o1js#1989, which makes the reliance on
nightly inevitable.
It seems that --offline works out of the box.

Another note is that cargo vendor has been introduced to provide a
consistent build environment for all platforms and all users, including
the CI. The lockfile Cargo.lock is meant to provide a locked environment
with all the dependencies, including the transitive ones. However, it is
only used by binaries. In our build process, we do build a static
library (libwires_15_stubs.a), not a binary. And Rust doesn't use the
lockfile in this case.
Another solution than `cargo vendor` would have been to strictly enforce
the version in the top-level Cargo.toml, but it implies to add all the
transitive dependencies (as otherwise, when a new transitive dependency
is released, cargo will fetch it, making the environment inconsistent).
This solution is not viable as the purpose of Cargo.toml is to enforce
versions on direct dependencies only, and leave the cargo solver doing
its job for the transitive dependencies.
For these reasons, the solution of vendoring the dependencies to build
the static library has been selected.
dannywillems added a commit to MinaProtocol/mina that referenced this issue Jan 28, 2025
@dannywillems
kimchi-bindings/js: back on nightly and use 2023-09-18
1870f9f 
nightly has been initially removed as it seemed to make vendoring all the
dependencies more complicated (which seems to not be the case though).
However, removing nightly introduces issues like "DataCloneError", see
o1-labs/o1js#1989, which makes the reliance on
nightly inevitable.
It seems that --offline works out of the box.

Another note is that cargo vendor has been introduced to provide a
consistent build environment for all platforms and all users, including
the CI. The lockfile Cargo.lock is meant to provide a locked environment
with all the dependencies, including the transitive ones. However, it is
only used by binaries. In our build process, we do build a static
library (libwires_15_stubs.a), not a binary. And Rust doesn't use the
lockfile in this case.
Another solution than `cargo vendor` would have been to strictly enforce
the version in the top-level Cargo.toml, but it implies to add all the
transitive dependencies (as otherwise, when a new transitive dependency
is released, cargo will fetch it, making the environment inconsistent).
This solution is not viable as the purpose of Cargo.toml is to enforce
versions on direct dependencies only, and leave the cargo solver doing
its job for the transitive dependencies.
For these reasons, the solution of vendoring the dependencies to build
the static library has been selected.
dannywillems added a commit to MinaProtocol/mina that referenced this issue Jan 28, 2025
@dannywillems
kimchi-bindings/js: back on nightly and use 2023-09-18
8b3eced 
nightly has been initially removed as it seemed to make vendoring all the
dependencies more complicated because some std lib must be included.
However, removing nightly introduces issues like "DataCloneError", see
o1-labs/o1js#1989, which makes the reliance on
nightly inevitable.
Removing --offline because stdlib missing for now (like `compiler_builtins`).

Another note is that cargo vendor has been introduced to provide a
consistent build environment for all platforms and all users, including
the CI. The lockfile Cargo.lock is meant to provide a locked environment
with all the dependencies, including the transitive ones. However, it is
only used by binaries. In our build process, we do build a static
library (libwires_15_stubs.a), not a binary. And Rust doesn't use the
lockfile in this case.
Another solution than `cargo vendor` would have been to strictly enforce
the version in the top-level Cargo.toml, but it implies to add all the
transitive dependencies (as otherwise, when a new transitive dependency
is released, cargo will fetch it, making the environment inconsistent).
This solution is not viable as the purpose of Cargo.toml is to enforce
versions on direct dependencies only, and leave the cargo solver doing
its job for the transitive dependencies.
For these reasons, the solution of vendoring the dependencies to build
the static library has been selected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dannywillems