`wit-bindgen` async directives are not composable

[rvolosatovs]

Feature

For the most part, whether or not a function should be generated as async should be dictated by WIT (refs #11246), however wasmtime-wasi might require generating some bindings as async even if they are not marked as such in WIT. See https://github.com/bytecodealliance/wasip3-prototyping/blob/3ce9ccb3efd3d15dc3ff55e31d0b85869847cf3f/crates/wasi/src/p3/bindings.rs#L21-L62 for a full list of bindings generated as async in wasip3-prototyping. An example of a use case is e.g. wasi:sockets implementation, which relies on a (Rust) async socket address check hook https://github.com/bytecodealliance/wasip3-prototyping/blob/3ce9ccb3efd3d15dc3ff55e31d0b85869847cf3f/crates/wasi/src/p3/sockets/mod.rs#L52-L58 https://github.com/bytecodealliance/wasip3-prototyping/blob/3ce9ccb3efd3d15dc3ff55e31d0b85869847cf3f/crates/wasi/src/p3/sockets/host/types/tcp.rs#L220

We need a way to "inherit" async directives from the WIT, while being able to "mark" additional ones as async

Benefit

This will drastically cut down the amount of boilerplate required to generate bindings, since currently passing only_imports to async bindgen argument is the only way to choose imports to be async. Note, that at least with current implementation, users of bindgen relying on generated bindings using with, e.g. embedders extending wasmtime_wasi with additional bindings MUST inherit the complete only_imports list from bindings depended upon (e.g. https://github.com/rvolosatovs/wasmtime/blob/c822293b26a1f2720d5a4cd035a59a66d4512e90/crates/wasi/src/p3/bindings.rs#L36-L83), otherwise compilation fails with:

error[E0277]: the trait bound `D: HostConcurrent` is not satisfied
   --> crates/wasi/src/p3/bindings.rs:20:1
    |
8   | / wasmtime::component::bindgen!({
9   | |     inline: "
10  | |         package example:wasi;
...   |
29  | |     concurrent_imports: true,
30  | | });
    | |__^ unsatisfied trait bound
    |
    = help: the trait `wasmtime_wasi::p3::bindings::sockets::types::HostConcurrent` is not implemented for `D`
note: required by a bound in `wasmtime_wasi::p3::bindings::sockets::types::add_to_linker`
   --> /Users/rvolosatovs/src/github.com/bytecodealliance/wasmtime/crates/wasi/src/p3/bindings.rs:73:5
    |
73  | /     wasmtime::component::bindgen!({
74  | |         path: "src/p3/wit",
75  | |         world: "wasi:cli/command",
...   |
125 | |         },
126 | |     });
    | |______^ required by this bound in `add_to_linker`
    = note: the full name for the type has been written to '/var/folders/bq/thy1_b7x29l7s2wqw39r62yw0000gn/T/rustdoctestObmR6k/rust_out.long-type-7090683266425947309.txt'
    = note: consider using `--verbose` to print the full type name to the console
    = note: this error originates in the macro `wasmtime::component::bindgen` (in Nightly builds, run with -Z macro-backtrace for more info)
help: consider further restricting type parameter `D` with trait `HostConcurrent`
    |
30  | }), D: wasmtime_wasi::p3::bindings::sockets::types::HostConcurrent;
    |   ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Implementation

The simplest approach that comes to mind is something like:

• rename async.only_imports -> async.imports (same for exports)
• introduce inherit: true|false, which is true by default and, when set to true would internally populate imports and exports lists with functions marked as async in WIT.
• marking a function as async twice (e.g. via async directive in WIT and by specifying it explicitly in imports) is not an error, but could show a warning/info statement if there was a verbose/debugging mode for the macro

Alternatives

we could require bindgen macro users who need extensibility to only rely on only_imports and such, but that seems pretty tedious.

add_imports or something like that, which would be mutually exclusive with only_imports is another option, but IMO just imports would be the least surprising option

inherit_exports and inherit_imports split is also possible, but IMO for use cases so niche, users should simply not use the "inheritance" at all and just manually specify the function names

[badeend]

In case it makes your life easier: all the problematic wasi-sockets IP address hooks can IMO be removed or refactored such that the WIT async-ness matches the generated bindings' async-ness:

• [method]tcp-socket.receive: Don't know why this needs a socket address hook. This method can only be called (successfully) after the TCP socket is already connected, at which point the hook will have already been called through either connect or listen.
• [method]tcp-socket.listen: the method itself doesn't need to be async. It can perform the permission check inside the already-async accept loop task, before doing the first actual accept call. This way, there's only one mode of async-ness from the guest's POV.
• [method]tcp-socket.bind & [method]udp-socket.bind: Maybe these do not need a hook at all. bind itself does not gain the guest any capability to do I/O. It's the follow-up calls (listen or connect for TCP, send or receive for UDP) that actually perform I/O, and those are already marked async in WIT.
• [method]udp-socket.connect: Does not perform I/O, so doesn't need permission checking. It's only the future calls to send & receive that need permission checking, and those are already async.

[rvolosatovs]

Thanks for the analysis @badeend !
Another reason we need the "artificial" async right now is the fact that we can't actually create stream or future handles from within sync bindings, so that's what drives the need for async receive, for example