Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement WASIX #328

Merged
merged 89 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
01fdb2a
Introduced our VirtualTaskManager and ModuleCache implementations
Aug 10, 2023
9bf94ac
Introduced the networking stack
Aug 10, 2023
003fb42
Copied the runtime across
Aug 11, 2023
1870cd4
The runtime works!
Aug 11, 2023
324d995
Adding Node back into CI
Aug 11, 2023
38a192d
Got the package building again
Aug 11, 2023
2e1576d
Rustfmt
Aug 11, 2023
e264aae
Added a CI step that generates API docs
Aug 11, 2023
98979f6
Remove the "package" script from package.json
Aug 11, 2023
626b678
Added TTYs
Aug 11, 2023
714e03e
Temporarily publish to GitHub Pages on every push
Aug 11, 2023
4d82353
Wired up a basic run() function
Aug 14, 2023
4e817da
Added a trivial test for run()
Aug 14, 2023
6818ce1
Trying to make the run() tests work on NodeJS
Aug 14, 2023
d10f97c
Introduced a `Wasmer` facade
Aug 14, 2023
ca7a8bb
Re-working the test suite to use Modern Web's browser test runner
Aug 14, 2023
0d6f8bd
Fleshing out the integration tests
Aug 15, 2023
7e2f6c9
More troubleshooting
Aug 16, 2023
7b87ce7
Working on my own threadpool implementation
Aug 21, 2023
c5dddc8
Debugging the thread pool
Aug 21, 2023
48cbb11
Stubbed out caching
Aug 21, 2023
6912428
Fixed most thread-safety issues
Aug 23, 2023
e072ed0
Wired up stdin/stdout/streams to JS
Aug 24, 2023
4c96bf7
Cleaning up the *.d.ts file a bit
Aug 24, 2023
79e556a
Wired up workers using spawn_with_module()
Aug 28, 2023
e5ef81e
Picked up wasmerio/wasmer#4186
Aug 28, 2023
293a408
Added a PackageLoader that always makes HTTP requests and relies on t…
Aug 29, 2023
7618ce6
Split things out into explicit actors
Aug 30, 2023
4c914f1
Added tests for waiting on an instance to exit
Aug 31, 2023
a4b2805
Make sure we interpret exit codes correctly
Aug 31, 2023
89d4a8e
Bump the wasmer dependency
Sep 11, 2023
2237dd3
Add a version of wasmer.sh
Sep 12, 2023
1869b2c
Update how we fetch python.webc from the registry
Sep 27, 2023
3c40adc
Fixing how we import init() from @wasmer/wasix
Sep 27, 2023
e619914
Revert the "lib.ts" changes and inject node polyfills while building
Sep 27, 2023
8d3cabc
Wire up builds for the example
Sep 27, 2023
096fc0e
Wire up CORS headers for the dev server
Sep 27, 2023
9a88e01
Expose init() as a named export rather than the default
Sep 29, 2023
4297baf
Re-working the worker.js script
Sep 29, 2023
94f2e10
Found a bug in the init() glue function where we don't allow reusing …
Oct 2, 2023
b963e24
Create a JS Error directly instead of sub-classing it
Oct 3, 2023
e6c94b5
Bump wasmer versions
Oct 3, 2023
faa0322
Added support for "uses"
Oct 4, 2023
e62c2f0
Make sure to use the wasmer::Module From implementation which goes th…
Oct 10, 2023
7947de9
implementing task_wasm()
Oct 11, 2023
3ecaddb
Modified the rollup config so bundles generated in dev mode can be im…
Oct 12, 2023
981f9ce
Introduced a SchedulerChannel and a module for serializing/deserializ…
Oct 13, 2023
697ba00
Moved each message type into its own module and renamed WorkerState t…
Oct 16, 2023
18e42df
Implemented SchedulerMessage serialization
Oct 16, 2023
b2ae4cb
Found a deadlock in the way task_wasm was being handled
Oct 17, 2023
d4e349c
Renamed the SchedulerChannel to just "Channel"
Oct 17, 2023
acef454
Moved the Runtime into RunConfig
Oct 18, 2023
c8068d9
Cleaning up logs
Oct 18, 2023
0cce73a
Finished deserializing scheduler messages
Oct 18, 2023
13ffbce
Make sure the HTTP client passes in a reference to the threadpool so …
Oct 18, 2023
bcbbc7f
Wired up xterm.js's fit addon
Oct 18, 2023
4c260c5
Fixed up the xterm.js CSS
Oct 19, 2023
0d9cd0b
Updated to the latest wasmer-js-fixes commit
Oct 19, 2023
05ff81f
Switch to a different logger
Oct 20, 2023
e6e5cf3
Cleaned up the example code
Oct 20, 2023
08cd7d4
Added an integration test for communicating with a subprocess's stdin
Oct 20, 2023
32d88c2
Updated to the spawn_exec commit
Oct 20, 2023
fcbeb59
Reverted the BYOB stream reader optimisation to avoid the possibility…
Oct 24, 2023
155ef46
Small fixes
Oct 24, 2023
d0370ff
Automatically wire up the TTY when running a command in interactive mode
Oct 24, 2023
d649d9c
Make copyStream sync
syrusakbary Oct 24, 2023
43695ea
Temporarily skip the "communicate with subprocess" test
Oct 25, 2023
760e7c4
Fix the TTY stdin stream to detect EOF instead of triggering an infin…
Oct 31, 2023
2e6425f
Give the PackageLoader a simple in-memory cache
Oct 31, 2023
b561609
Wire up Container::from_registry() and make it go through the Package…
Oct 31, 2023
19847ee
Debugging timeouts in the TypeScript integration tests
Oct 31, 2023
b23623b
Re-worked the tests to use quickjs instead of python
Nov 2, 2023
07e7962
Make logger initialization a separate operation
Nov 2, 2023
61f8371
Tweak streaming to respect chunk sizes
Nov 2, 2023
008cbb0
Closing stdin manually
Nov 2, 2023
f0e5703
More stream debugging
Nov 6, 2023
928b251
Switching tests over to christoph/wasix-test-stdinout
Nov 7, 2023
2b17c18
Skipping all but one failing test
Nov 7, 2023
37c1864
Switching the logger back to tracing-wasm
Nov 10, 2023
f452703
Move logging to its own module and forward writes directly to `consol…
Nov 13, 2023
26ec0a7
We don't care if the user already closed stdin
Nov 15, 2023
03023b3
Created our own logger implementation
Nov 15, 2023
7fbbaba
Formatted the wasmer.sh example
Nov 15, 2023
6772d5f
Got the QuickJS integration test working
Nov 15, 2023
42b53d9
Skipping some tests temporarily
Nov 15, 2023
5fa77a2
Cleaned up test output
Nov 15, 2023
8b5d439
Got the Python TTY test working
Nov 17, 2023
49c4652
Skip the failing TTY tests
Nov 21, 2023
0a62cac
Use a debug build when testing in CI
Nov 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[build]
target = "wasm32-unknown-unknown"

[target.'cfg(target_arch = "wasm32")']
# Use "wasmer" for running tests when compiled to WebAssembly
runner = ["wasmer", "run"]
# This is needed so the module is compiled with atomics support (shared memory)
# We add the `-no-check-features` linker args because otherwise one of the modules fails to link
rustflags = '-Ctarget-feature=+atomics,+bulk-memory -Clink-args=--no-check-features --cfg=web_sys_unstable_apis'

[unstable]
# We want to make sure std gets built with atomics, too
build-std = ['std', 'panic_abort']
86 changes: 75 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ on:
branches:
- main

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
check:
name: Compile and Test
Expand All @@ -16,22 +20,82 @@ jobs:
uses: actions/setup-node@v3
with:
node-version: 16
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: "1.64.0" # Required for [workspace.package]
override: true
target: wasm32-unknown-unknown
- name: Setup Rust
uses: dsherret/rust-toolchain-file@v1
- name: Install Nextest
uses: taiki-e/install-action@nextest
- name: Install wasm-pack
uses: taiki-e/install-action@wasm-pack
- name: Install wasm-strip and wasm-opt
run: sudo apt-get update && sudo apt-get install -y wabt binaryen
- name: Rust Cache
uses: Swatinem/rust-cache@v2
- name: Type Checking
run: cargo check --workspace --verbose --locked
- name: Install JS Dependencies
run: npm install
- name: Build
run: npm run build:dev
- name: Test
run: npm run test

api-docs:
name: API Docs
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Node
uses: actions/setup-node@v3
with:
node-version: 16
- name: Setup Rust
uses: dsherret/rust-toolchain-file@v1
- name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
uses: taiki-e/install-action@wasm-pack
- name: Install wasm-strip and wasm-opt
run: sudo apt-get update && sudo apt-get install -y wabt binaryen
- name: Install Dependencies
- name: Rust Cache
uses: Swatinem/rust-cache@v2
- name: Install JS Dependencies
run: npm install
- name: Build
run: npm run build
- name: Test
run: npm run test
- name: Generate Docs
run: npx typedoc --out docs lib.ts
- name: Upload API Docs
uses: JamesIves/[email protected]
# FIXME: Re-enable this before merging
# if: github.ref == 'refs/heads/main'
with:
branch: gh-pages
folder: docs
single-commit: true

lints:
name: Linting and Formatting
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Rust Cache
uses: Swatinem/rust-cache@v2
- name: Setup Rust
uses: dsherret/rust-toolchain-file@v1
- name: Check Formatting
run: cargo fmt --all --verbose --check
- name: Clippy
run: cargo clippy --workspace --verbose

workflow-times:
name: Workflow Timings
runs-on: ubuntu-latest
needs: check
steps:
- name: Time Reporter
uses: Michael-F-Bryan/[email protected]
with:
token: ${{ secrets.GITHUB_TOKEN }}
jobs: Compile and Test
message: |
Make sure you keep an eye on build times!

The goal is to keep CI times under 10 minutes so developers can maintain a fast edit-compile-test cycle.
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
/bin
/target
target/
**/*.rs.bk
node_modules/
dist/
package-lock.json
*.log
pkg/
docs/
Loading
Loading