Improve Lind Toolchain docs #231

lukpueh · 2025-05-27T09:45:02Z

Current documentation about the Lind toolchain used for building and running Lind programs can be found in several docs under: https://github.com/Lind-Project/lind-wasm/tree/997a29c89f1d26e400f7f18eca062914f78c62c9/docs/build

Some issues with the current documentation include:

lack of bigger picture (how do these tools relate?)
outdated
overly detailed (e.g. libc)
overly succinct (e.g. bazel)

This patch adds a new single document, which gives an overview of the full Lind toolchain, and how it is built step by step, with references to Internal Documentation for background information about the toolchain, and build scripts (Docker, bazel, shell scripts) for detailed build commands and options.

Note that I deliberately added the document under the Usage section, because I think it could fully replace all documents in the Build System section, and then probably wouldn't need its own top-level section on the website. Moreover, the Usage section seems like a good fit given the current experimental state of the project, where usage very likely involves tinkering with the toolchain.

Note to reviewer: Please check..

correctness
degree of detail (too much? too little?)
typography (too much switching between bold, italic, code, link markup?)
links

See our add docs guide for how to build the docs site locally.

related #180 #196

Current documentation about the Lind toolchain used for building and running Lind programs can be found in several docs under: https://github.com/Lind-Project/lind-wasm/tree/997a29c89f1d26e400f7f18eca062914f78c62c9/docs/build Some issues with the current documentation include: - lack of bigger picture (how do these tools relate?) - outdated - overly detailed (e.g. libc) - overly succinct (e.g. bazel) This patch adds a new single document, which gives an **overview** of the full Lind toolchain, and how it is built step by step, with references to Internal Documentation for background information about the toolchain, and build scripts (Docker, bazel, shell scripts) for detailed build commands and options. Note that I deliberately added the document under the *Usage* section, because I think it could fully replace all documents in the *Build System* section, and then probably wouldn't need its own top-level section on the website. Moreover, the *Usage* section seems like a good fit given the current experimental state of the project, where usage very likely involves tinkering with the toolchain. Signed-off-by: Lukas Puehringer <[email protected]>

ci-response-bot · 2025-05-27T09:49:46Z

Commit 254f831: Build Failed

View Log

lukpueh

Can someone fill the gaps below? cc @Yaxuan-w, @rennergade, @qianxichen233

You can comment here and I add it to the PR. Or we merge first, and edit in new PRs.

lukpueh · 2025-05-27T11:17:29Z

docs/use/toolchain.md

+
+3. __Build *glibc* and generate sysroot__ (see `make_glibc` in [BUILD](https://github.com/Lind-Project/lind-wasm/blob/main/BUILD) file)
+    1. Configure and compile *glibc* for the *WASI* target with *Clang*.  (see
+      [`wasm-config.sh`](https://github.com/Lind-Project/lind-wasm/blob/main/src/glibc/wasm-config.sh))


Let's adds some information about the used build flags here, like the current build glibc doc does, but with better explanation of why these flags are are used with the given values.

Most notably, why do we use "i686-linux-gnu" as --host and --build argument?

@qianxichen233

lukpueh · 2025-05-27T11:20:17Z

docs/use/toolchain.md

+        - `nptl/pthread_create.c`
+        - `lind_syscall/lind_syscall.c`
+        - `csu/wasm32/wasi_thread_start.s`
+        - `csu/wasm32/set_stack_pointer.s`


It would be great to add some extra information here, like a oneliner description of these files and why they need to be compiled separately?

I think it would also be worth to describe or give an overview of the used compiler flags. Most notably, what's with all the includes?

lind_syscall/lind_syscall.c: This file serves as the interface in glibc to redirect syscalls into Wasmtime. It’s specific to our lind-wasm setup and handles the syscall transition logic.

nptl/pthread_create.c, csu/wasm32/wasi_thread_start.s, csu/wasm32/set_stack_pointer.s: These three are related to our thread/multi-process support in lind-wasm. Some of the changes we made for enabling thread functionality involve these files.

I think @qianxichen233 could provide more detailed context here

Maybe listing those files explicitly is too much detail for an overview-type document. Would it make sense to just say, that we compile a few extra files here, which are not part of glibc, but are needed in the sysroot for reasons XZY? Could you help with phrasing?

Maybe we could provide more details in the internal/libc.md document.

rennergade · 2025-05-27T16:17:31Z

Thanks @lukpueh !

I tagged @qianxichen233 here, I think he has the best answers for the specific questions you asked. Also tagging @m-hemmings to review.

m-hemmings · 2025-06-03T16:18:51Z

docs/use/toolchain.md

+  in lind programs
+- A custom *WebAssembly* runtime ([`wasmtime`](../internal/wasmtime.md)) based
+  on [*RawPOSIX*](../internal/rawposix.md) to run lind programs
+- Rust Cargo to build `wasmtime`


Is it common to refer to it as "Rust Cargo" or just "Cargo"?

I wanted to emphasize the multi-lang nature of the project. But you're right, Rust Cargo sounds odd. What about Rust/Cargo? :D I'm also fine with just Cargo.

It's a nit to pick, so whatever makes sense is fine with me, I can certainly appreciate wanting to emphasize that Cargo is a Rust tool

m-hemmings · 2025-06-03T16:20:40Z

docs/use/toolchain.md

+  and lind programs
+- A custom [*glibc*](../internal/libc.md) used as sysroot to build
+  lind programs
+- A custom `wasm-opt` binary to enable [multi-processing](../internal/multiprocess-support.md)


is the method of customization of either this or the following custom WebAssembly explained anywhere? Is it important to?

Good point! I think the internal/* documentation should and in parts already does elaborate on this.

Maybe we can link to that in a footnote or something on this and expand on it in the appropriate place

I think there's a bit more to this to do with signaling, hopefully @qianxichen233 can clarify.

Maybe we can link to that in a footnote or something on this and expand on it in the appropriate place

The word "multi-processing" here links to the internal documentation. Do you prefer a footnote? Or a different link text?

I can see how outlining just multi-processing, could look like it links to a general explanation of multi-processing. Adding the link to wasm-opt might be more helpful. The problem is, we currently don't mention wasm-opt in that text at all.

What do you think about moving the link to wasm-opt and create an issue for mentioning it in the mutli-processing doc?

What do you think about moving the link to wasm-opt and create an issue for mentioning it in the mutli-processing doc?

Just changed the link, will post an issue shortly.

docs/use/toolchain.md

Yaxuan-w · 2025-06-04T15:53:06Z

docs/use/toolchain.md

+        - `nptl/pthread_create.c`
+        - `lind_syscall/lind_syscall.c`
+        - `csu/wasm32/wasi_thread_start.s`
+        - `csu/wasm32/set_stack_pointer.s`


lind_syscall/lind_syscall.c: This file serves as the interface in glibc to redirect syscalls into Wasmtime. It’s specific to our lind-wasm setup and handles the syscall transition logic.

nptl/pthread_create.c, csu/wasm32/wasi_thread_start.s, csu/wasm32/set_stack_pointer.s: These three are related to our thread/multi-process support in lind-wasm. Some of the changes we made for enabling thread functionality involve these files.

I think @qianxichen233 could provide more detailed context here

Reword based on review: - Clarify wasmtime/rawposix relationship - relocate link text to clarify what is linked to - Remove redundancy: "Rust Cargo" -> "Cargo" Signed-off-by: Lukas Puehringer <[email protected]>

ci-response-bot · 2025-06-05T10:17:36Z

Commit 674faeb: Build Failed

View Log

lukpueh · 2025-06-16T18:16:49Z

Thanks for your comments. This is ready for another round of review

A replacement was added with Lind-Project#231. Closes Lind-Project#232 (see issue description for details) Signed-off-by: Lukas Puehringer <[email protected]>

lukpueh commented May 27, 2025

View reviewed changes

lukpueh mentioned this pull request May 27, 2025

Consider removing *Build System* docs #232

Closed

rennergade requested review from m-hemmings and qianxichen233 May 27, 2025 16:16

rennergade requested review from rennergade and Yaxuan-w June 2, 2025 18:18

m-hemmings reviewed Jun 3, 2025

View reviewed changes

lukpueh mentioned this pull request Jun 4, 2025

Review CI platforms and build tools #235

Closed

Yaxuan-w reviewed Jun 4, 2025

View reviewed changes

Minor rewords in toolchain docs

674faeb

Reword based on review: - Clarify wasmtime/rawposix relationship - relocate link text to clarify what is linked to - Remove redundancy: "Rust Cargo" -> "Cargo" Signed-off-by: Lukas Puehringer <[email protected]>

lukpueh requested review from m-hemmings and Yaxuan-w June 16, 2025 08:20

lukpueh mentioned this pull request Jun 16, 2025

docs: explain how wasm-opt and wasmtime are customized #241

Open

Yaxuan-w approved these changes Jun 18, 2025

View reviewed changes

rennergade approved these changes Jun 19, 2025

View reviewed changes

rennergade merged commit 2b6913e into Lind-Project:main Jun 19, 2025
1 check failed

lukpueh added a commit to lukpueh/lind-wasm that referenced this pull request Jun 25, 2025

Remove Build System docs

77f8ea1

A replacement was added with Lind-Project#231. Closes Lind-Project#232 (see issue description for details) Signed-off-by: Lukas Puehringer <[email protected]>

lukpueh mentioned this pull request Jun 25, 2025

Remove Build System docs #248

Merged

Improve Lind Toolchain docs #231

Improve Lind Toolchain docs #231

Uh oh!

Conversation

lukpueh commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ci-response-bot bot commented May 27, 2025

Uh oh!

lukpueh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yaxuan-w Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rennergade commented May 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Yaxuan-w Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ci-response-bot bot commented Jun 5, 2025

Uh oh!

lukpueh commented Jun 16, 2025

Uh oh!

Uh oh!

Uh oh!

lukpueh commented May 27, 2025 •

edited

Loading

Yaxuan-w Jun 4, 2025 •

edited

Loading

Yaxuan-w Jun 4, 2025 •

edited

Loading