Split 'use' inside worlds into 'use import' and 'use export'

[lukewagner]

This PR proposes to change how use works inside WIT worlds, based on some initial discussion in wit-bindgen/#822.

Currently, use can be used with the same syntax in both interfaces and worlds. For interfaces, the syntax works great, but in a world context, it's rather ambiguous whether a use refers to imports or exports (and when interfaces are implicitly pulled in, whether they are pulled in as imports or exports). Furthermore, in the advanced case where you want to both import and export the same interface and be able to refer to both imported and exported versions of a type defined in that interface, it's not possible. (It is expressible in Component Model WAT, though.)

This PR avoids the ambiguity and removes the expressivity gap by removing the plain use x syntax from worlds and, in its place, adding use import x and use export x. See the PR for an example.

This change is only at the WIT-level; Component Model WAT can already express both of these more-explicit versions of use (via alias definitions). I'm not aware of any WASI WIT that actually uses use in a world, so this doesn't affect the literal WIT stabilized in 0.2, which is nice. It might break some handwritten worlds though (see the abovementioned wit-bindgen issue), so perhaps we could support (but warn for) the existing use-in-worlds syntax for a period of time.

[alexcrichton]

Thanks for writing this up!

I'd be hesitant to merge this as-is as I suspect it'll take a bit to get this implemented and I'd want to avoid the case where the docs here don't reflect the state of the world. That being said this is probably going to be a common problem, so I feel like we should fix this at some point. I know we have the emoji bits but holding everything up or requiring things purely because of an implementation doesn't feel great. If this isn't where most people come for docs, though, then it's probably ok if the two drift. Given where we are in the proposal though I feel like folks still come here pretty frequently to double-check behaviors and such.

From a more bike-sheddy perspective it feels a bit unfortunate that use is sort of inconsistent between an interface and a world now. Not only syntactically but also sort of semantically, and I'm not sure how we can reflect this. For example in a world you can pick to import from either an import or an export, but for an interface you're not given that choice (sort of rightfully so). Technically though with an exported interface you could select from either another exported version of the dependency or an imported version.

For the syntactic difference we could perhaps have import use ... and export use ... to reverse the order of the keywords, but I'm not sure what to do about having an interface also having the choice here (it feels like an interface both should and should not have the ability to pick an import/export).

In terms of breakage I don't think we can remove the existing use form in worlds. I think we should document it as a "sugared form" of use import and then in the future consider trying to deprecate it.

[lukewagner]

Thanks for all the feedback! I'm happy to not merge this PR until the new syntax being proposed has been implemented so that if people see it in WIT.md and then write it using updated release tooling, it works. We could go the emoji-tag route, but seeing as how this isn't a fundamental C-M feature, but rather a WIT syntax change, it feels like maybe we could avoid the overhead.

It also makes sense in a transitional time frame to continue to parse and support the existing use-in-world syntax, considering it deprecated (and perhaps emitting a warning).

Regarding the asymmetry between interface and world: I think this asymmetry is justified and it follows from the fact that an interface can necessarily show up in both imports or exports of a component/world and thus interfaces have to be agnostic to their "direction". For the use case you mention where maybe I want to export an interface and wire up a resource type that it uses to either an import or an export, I think the solution here isn't to add anything to interface (which, having to be agnostic to whether it is imported or exported, would have a hard-if-not-impossible time saying what it wants in isolation) but rather have syntax to override the default behavior when the interface is imported or exported in a particular containing world (noting that a single interface may legitimately get wired up in several different ways in several different worlds). So, e.g., I was imagining a syntax like:

interface i { resource r; }
interface j { use i.{r}; f: func() -> r; }
world w {
  import i;
  export i;
  export j with { i = import i };
}

to override what, iirc, is the default behavior that the export j would otherwise use the r of the exported i. This may not be the right syntax, but does the idea make sense? If so, from this example, it seems like import|export is in effect part of the fully-qualified name of an interface in the context of a world which then explains why use import/use export look the way they do.

[alexcrichton]

Ok yeah I'm sold on that point, and that sounds good to me. I like the idea of purposing with after an exported interface for this, should work! (I also feel like we've talked about this before and I'm showing how bad my memory is)

How do you feel about the syntax bikeshed of use import vs import use?

[lukewagner]

I don't feel super-strongly, but I suppose I still have a preference for use import because:

• You can think of use import foo.{r} as use followed by the name of what to project from, import foo, and this lines up with the idea that import foo is the fully-qualified name that, e.g., you can use in the above with { foo = import foo } syntax.
• import ____ suggests to me that we're emitting an import, which may end up actually happening as a side effect, but it seems like the primary thing a use is doing is projecting something out of an interface.

Just sketching an example both ways

world w {
  import wasi:http/outgoing-handler;
  use import wasi:http/types.{request, response};
  export frob: func(r: request) -> response;
}

vs.

world w {
  import wasi:http/outgoing-handler;
  import use wasi:http/types.{request, response};
  export frob: func(r: request) -> response;
}

subjectively, the former seems to more clearly explain to me that we're doing 3 distinct things, importing, projecting and then exporting rather than doing 2 imports and 1 export.

[alexcrichton]

Ok yeah I'm sold on that as well 👍

[squillace]

the only thing I'd add after this while, finding this discussion is that it's import and export that are hard to understand for people intuitively as "direction" of consumption or production. If you then use that word set prior to use I think it's less understandable for most. Not sure that's helpful, but it tends in the direction you're going here.

[rylev]

I'm interested in this functionality as it's necessary for some virtualization scenarios I'm working on. I'd be interested in potentially helping implement this in the various tools. However some thoughts/questions:

Bikeshedding use syntax

Looking at the current examples, I'm interested in why the use would still be necessary in worlds considering that we're already breaking symmetry with interfaces:

Rewriting the example @lukewagner gave without the use keyword makes things clearer in my opinion. We're simply importing those items which naturally implies they're in scope to be used by other definitions.

world w {
  import wasi:http/outgoing-handler;
  import wasi:http/types.{request, response};
  export frob: func(r: request) -> response;
}

In my mind this is also easier to learn as newcomers are not confronted with a new keyword. What does the use keyword here actually afford us?

** Bikeshedding with syntax **

My use case requires the ability to specify whether a use inside of an interface is supplied by an import or an export so I would very much like to see the with syntax fleshed out here in the proposal.

The proposed syntax from @lukewagner seems reasonable:

export j with { i = import i };

Though I can imagine a few variants:

export j { i = import i };

Perhaps the with keyword gives us the flexibility to use naked {} in the future for something else, but I don't necessarily think it's strictly necessary for readability today.

export j using { i = import i };

The use of a using keyword contrasts nicely with the use in the interface itself, but I suppose there is precedent to using with for further qualifying items (like in the include statement).

In any case, @lukewagner did you want to take a stab at writing the grammar for this disambiguation syntax before we start down the path of implementation?

[lukewagner]

@rylev Great to hear and great feedback! On first consideration, I really like your proposed alternative to kill use and stick the projects on the import directly (which does indeed read nicely). And to your second point, I'm not tied to with, but I do feel like a keyword separator is useful (both for syntactic forward-compatibility and to help the reader know what this curly-block is doing) and using does have a nice symmetry with use, so I'm up for that too. I'll update the grammar.

[lukewagner]

On second thought, working on the grammar, I think maybe having import statements also provide resource-type projection (instead of having a separate use statement for this) leads to some rather confusing compound statements. E.g.:

world w {
  ...
  export wasi:http/types.{request} using { wasi:io/error = my-error };
  export foo: interface {
    resource res;
    foo: func() -> res;
  }.{res};
  export frob: func(r: res) -> request;
}

Although less compact, with a separate use, this would look like:

world w {
  export wasi:http/types using { wasi:io/error = my-error };
  export foo: interface {
    resource res;
    foo: func() -> res;
  };
  use export foo.{res};
  use export wasi:http/types.{request};
  export frob: func(r: res) -> request;
}

WDYT?

[rylev]

@lukewagner I'm convinced. Your example with use is certainly easier to read than then one that attempts to get rid of use completely.

[squillace]

agree quickly with @rylev here; having the explicit use is designed to ensure complete intelligibility and not be perfect for human readers, thought that's what we have now. I like the second version.

[squillace]

OH! one other thing. In past "interface sharing technologies" there was much too and fro about having a shared typelib of interfaces that both sides could reuse; it was the only thing that made "getting it right on the wire" possible for humans. In this case, we're NOT trying to nail the wire protocol down to the bit/endianess, but the general usage problem remains. I see this as paving that path to shared types. If I'm mistaken, def let me know. :-)

[lukewagner]

@rylev Ok, cool, I left the use import/export syntax as-is, but I added the syntax for the using clause in this new commit since it sounded like you wanted using in this unit of work.