Editorial: Reduce Annex B's monkey-patching

[jmdyck]

See Issue #1595 for some background. However, that talks about making many Annex B things normative, which this PR doesn't do. This PR takes pseudocode-replacements from Annex B and splices them into the corresponding points in the main body, while maintaining their normative-optional status.

This PR doesn't eliminate all Annex B monkey-patching, because I'm avoiding Annex B features that modify the grammar (due to controversy), i.e.:

• all of B.1 Additional Syntax
• B.3.3 FunctionDeclarations in IfStatement Statement Clauses
• B.3.5 Initializers in ForIn Statement Heads

(It's unclear why the latter two aren't included in B.1, since they define additional syntax.)

That leaves:

• B.3.1 Labelled Function Declarations
• B.3.2 Block-Level Function Declarations Web Legacy Compatibility Semantics
• B.3.4 VariableStatements in Catch Blocks
• B.3.6 The [[IsHTMLDDA]] Internal Slot

This PR has one commit for each of those sections, plus an extra one at the start to make B.3.2 easier (which might be useful to cherry-pick, even if this PR goes nowhere). This PR is also 'based' on PR #2951, so that'll show up as a commit here until it's merged.

---

Here's roughly the scheme I followed when eliminating a monkey-patch of an abstract operation. (There are also early error rules that get patched, but the edits are analogous, and simpler.)

Where the status quo has:

MainlineAlg:
  1. [id="step-id"] Do the mainline thing. NOTE: This step is replaced in section (Annex B feature).

Annex B feature:
  During MainlineAlg the following steps are performed in place of step (step-id):
    1. Do the Annex B thing.

this PR changes that to:

MainlineAlg:
  1. [id="step-id"] If <ins normative-optional>the host supports (Annex B feature)</ins>, then
    1. Do the Annex B thing.
  1. Else,
    1. Do the mainline thing.

Annex B feature:
  This feature involves special semantics at step (step-id) in MainlineAlg.

(In some cases, "Do the mainline thing" is empty, which simplifies things.)

Note that the above format of the "host supports" step is a compromise to get the current ecmarkup to accept and render it. What I had in mind instead was something more like:

  1. [id="step-id" normative-optional] If the host supports (Annex B feature), then

ecmarkup could then recognize the "normative-optional" step-attribute and apply appropriate styling for that step and all its substeps.

[jmdyck]

Note that, in the rendered spec's CSS, the normative-optional attribute forces its element to have display:block, which doesn't look great when the element isn't block-display to begin with, as is the case for my uses of <ins normative-optional> here.

I was thinking of dropping the normative-optional attribute just to get a nicer temp rendering, but now I'm not sure if I want to bother.

Anyhow, I stress that the build preview is not the final rendering; that will require some changes to ecmarkup.

[jmdyck]

(force-pushed to resolve conflicts from #2901 and #2915)

[michaelficarra]

We discussed this at editor call and decided this should be run by committee first so we can hear any objections. We still maintain that these changes are within editorial discretion, though.

Also, we will need to change the wording of the "feature tests" to make sure we aren't changing or obfuscating the web browser requirement.

[jmdyck]

Also, we will need to change the wording of the "feature tests" to make sure we aren't changing or obfuscating the web browser requirement.

I gather that the problem is that the wording "If the host supports [feature]" might be misread to give the host the freedom to support [feature] or not, whereas only non-browsers have that option (browsers must support it). And even though the start of Annex B makes that clear, and "[feature]" is a link to the feature description within Annex B, that might not be enough.

So one possibility would be to change the wording to something like "If the host is a web browser or the host supports [feature]".

Another might be to make the phrase "the host supports" be a link to the start of Annex B, and possibly make that a better landing site for that link.

[michaelficarra]

@jmdyck Correct, and yes those are some reasonable alternative strategies. Would you like the editors to discuss it more and give you a specific recommendation for the wording, or do you want to come up with something first?

[jmdyck]

Hm, not much left for me to come up with, other than elaborating on a "better landing site". Could be something like:

Each of the features defined in this annex is specified as a set of modifications to the syntax, semantics, and/or built-in properties specified in the main body of this document. In some cases, those modifications appear in this annex, and in others, they appear (specially marked) in the main body.

When a modification to semantics appears in the main body (in an algorithm or early error rule), it is guarded by the condition that "the host supports" the relevant feature. When the host is a web browser, this condition must be true. When the host is not a web browser, it is host-defined whether the condition is true or false.

[jmdyck]

I now realize that the 'normative-optional' attribute isn't right for these cases, so it should probably be something new, e.g. 'annex-b'. Let me know what you'd like.

[bakkot]

For the inline note, how about:

If the host is a web browser or otherwise supports [feature]

with a manual link to the "landing site" in annex B. And then the "landing site" wording you suggest as above is fine, except I'd tweak the last couple sentences as in

Each of the features defined in this annex is specified as a set of modifications to the syntax, semantics, and/or built-in properties specified in the main body of this document. In some cases, those modifications appear in this annex, and in others, they appear (specially marked) in the main body.

When a modification to semantics appears in the main body (in an algorithm or early error rule), it is guarded by the condition that "the host supports" the relevant feature. Web browsers are required to support all such features.

---

We're also fine with calling these normative-optional even though the "optional" part is not technically true for browsers.

[jmdyck]

For the inline note, how about:

If the host is a web browser or otherwise supports [feature]

I've done this, but I'm somewhat dissatisfied with the results. In particular, in some cases (in FunctionDeclarationInstantiation,
EvalDeclarationInstantiation, and all the early errors), the "supports" condition is the first term of a conjunction, so you now get things like:

If the host is a web browser or otherwise supports [feature] and strict is false, then

and

It is a Syntax Error if [...], unless the host is a web browser or otherwise supports [feature], the source text matched by this production is not strict mode code, and the duplicate entries are only bound by FunctionDeclarations.

I'm not saying it's an A or B and C ambiguity (precedence problem), but it is a bit clunky.

with a manual link to the "landing site" in annex B.

Instead of a manual link, I added variants="otherwise supports" to the <dfn> for "the host supports".

And then the "landing site" wording you suggest as above is fine, except I'd tweak the last couple sentences as in

[...] it is guarded by the condition that "the host supports" the relevant feature.

Although now we never use that phrase. But people can (in the rendered spec) hover the term and get all the references, so not a problem, I guess.

We're also fine with calling these normative-optional even though the "optional" part is not technically true for browsers.

I'm not fine with it, though. Calling them normative-optional would imply a change in normative status, and I want to be clear that this PR doesn't do that.

[jmdyck]

(force-pushed to get the updated enforce-format.yml)

[jmdyck]

For the inline note, how about:

If the host is a web browser or otherwise supports [feature]

[...]

I'm not saying it's an A or B and C ambiguity (precedence problem), but it is a bit clunky.

Now that I can see the rendered version, this bothers me less.

[bakkot]

Let's just tweak the definition of "normative optional" to add "unless otherwise indicated", then, as in

A conforming implementation of ECMAScript may choose to implement or not implement Normative Optional subclauses, unless otherwise indicated.

[bakkot]

We discussed this in plenary today, and while one delegate was not comfortable with it (specifically the function-in-block part) everyone else who expressed an opinion was strongly in favor, so we're going to go forward with it. Editors will still need to review for wording and so on, of course.

[jmdyck]

(force-pushed to resolve merge-conflicts from PR #2877)

[jmdyck]

Reminder: This PR is 'based' on PR #2951, so that PR should be resolved first.

(2951 moves one of the insertion-points for Annex B stuff, so that's easier to do before inlining than after.)

[nicolo-ribaudo]

What do you think about this rendering, but with the "ANNEX B" title linking to https://tc39.es/ecma262/#sec-additional-ecmascript-features-for-web-browsers?

[jmdyck]

Let's just tweak the definition of "normative optional" to add "unless otherwise indicated", then [...]

I'm not a fan of "unless otherwise indicated", but I'm not finding alternative wording that fits the context, so will add this.

And if the inlined bits of Annex B are going to be labelled normative-optional, I imagine that should apply to the whole of Annex B too. (I.e., change its <emu-annex> from normative to normative-optional.)

[ljharb]

It won’t necessarily apply to everything, some things may end up being required.

[gibson042]

Shouldn't this somehow restrict scope? Otherwise it's unclear what is meant by "a |Block|, |CaseClause|, or |DefaultClause|".

[jmdyck]

I agree. Without any qualification, it seems like "a |Block|, |CaseClause|, or |DefaultClause|" could refer to any such Parse Node that the implementation is aware of, which seems too broad.

Originally (in ES6), it was scoped via "For each |FunctionDeclaration| f in varDeclarations ...", but then that was removed in commit efbfc88, which doesn't appear to be connected to a PR, but claims to fix old bug 4427. However, if you look at that bug report's suggested fix, in addition to removing "in varDeclarations", it also says to append "Contained within code", and that part didn't happen in the commit.

For discussion that prompted the bug report, see esdiscuss.

Anyhow, that's a preexisting problem, so not the main point of this PR, but I could tack on a commit if the editors want.

[jmdyck]

On second thought, I figured it deserved its own PR: #3361.

[gibson042]

This renders poorly, and in particular misses the "normative optional" text: https://ci.tc39.es/preview/tc39/ecma262/sha/06e2af4a47bd749e5cbb0fb0c737221e86685a85/multipage/ecmascript-language-statements-and-declarations.html#sec-block-static-semantics-early-errors

I wonder if moving the attribute up a level would improve things.

Suggested change

	<li>
	It is a Syntax Error if the LexicallyDeclaredNames of |StatementList| contains any duplicate entries.
	It is a Syntax Error if the LexicallyDeclaredNames of |StatementList| contains any duplicate entries<span normative-optional>, unless IsStrict(this production) is *false*, the duplicate entries are only bound by FunctionDeclarations, and the host is a web browser or otherwise supports <emu-xref href="#sec-block-level-function-declarations-web-legacy-compatibility-semantics" title></emu-xref></span>.
	</li>
	<li>
	<div>It is a Syntax Error if the LexicallyDeclaredNames of |StatementList| contains any duplicate entries.</div>
	<div normative-optional>If the host is a web browser or otherwise supports <emu-xref href="#sec-block-level-function-declarations-web-legacy-compatibility-semantics" title></emu-xref>, the above Syntax Error does not apply when IsStrict(this production) is *false* and the duplicate entries are only bound by FunctionDeclarations.</div>
	</li>

[jmdyck]

This renders poorly, and in particular misses the "normative optional" text:

I agree that it renders poorly, but I'm not sure what you mean by "misses". The normative optional text is there, but the bottom of each line is slightly obscured/clipped by the padding of the line below.

ecmarkup.css probably needs a new rule for when normative-optional occurs on inline content. (Currently, that attribute is only attached to block content, specifically emu-clause elements.) E.g., if you reduce the padding from 0.5em to 0.2em, the clipping goes away. (Also, the border-left line maybe doesn't make sense for inline content.)

[gibson042]

I agree that it renders poorly, but I'm not sure what you mean by "misses". The normative optional text is there, but the bottom of each line is slightly obscured/clipped by the padding of the line below.

I mean that the literal text "NORMATIVE OPTIONAL" is absent, but Conformance indicates that it should be present:

A Normative Optional clause is denoted in this specification with the words "Normative Optional" in a coloured box, as shown below.

current

proposed

[jmdyck]

I mean that the literal text "NORMATIVE OPTIONAL" is absent

Ahh, gotcha.

[gibson042]

Likewise here (<li><p>It is a Syntax Error if…</p><p normative-optional>If the host is a web browser or otherwise supports…</p></li>).

[gibson042]

And here (etc.).

[syg]

Generally lgtm, some questions about wording.

[syg]

I find the "unless" too long now and hard to parse. WDYT splitting this, and all Syntax Error sentences, into:

If the host is not a web browser and does not otherwise supports <blah>, it is a Syntax Error if the LexicallyDeclaredNames of |StatementList| contains any duplicate entries.
If the host is a web browser or otherwise supports <blah>, it is a SyntaxError if the LexicallyDeclaredNames of |StatementList| contains any duplicate entries, unless IsStrict(this production) is *false*, the duplicate entries are only bound by FunctionDeclarations.

Thoughts @bakkot @michaelficarra?

[michaelficarra]

I think the second constraint you have there is mis-worded? Not sure what it's meant to say, but I think I get the idea and that would be fine by me.

[syg]

Oh I think I see your confusion. The structure I intended was:

• If the host does support this particular normative optional feature, (it is a Syntax Error if reasons, unless these other reasons).

Is your confusion that it can also be parsed as the following?

• (If the host does support this particular normative optional feature, it is a Syntax Error if reasons), unless these other reasons.

[michaelficarra]

Yes.

[syg]

Yeah I dunno then, I don't see a good way to split it up otherwise. It's hard to actually move the entire unless to the front as an if because the unless has an anaphora ("the duplicate entries").

[michaelficarra]

Then leave it as-is?

[jmdyck]

Re the "unless" being too long and hard to parse, we could introduce a <ul>. Something like:

It is a Syntax Error if (duplicates), unless:
- IsStrict(this production) is *false*,
- the duplicate entries are only bound by FunctionDeclarations, and
- the host is a web browser or otherwise supports (link).

[syg]

A ul wfm.

Please have it say either "unless all the following are true", or have a trailing ", and" on every non-final bullet.

[jmdyck]

Okay, I've added a fixup commit to introduce a <ul> into this rule and also a similar one further down.

There are some others that only have a 2-term conjunction; I'm assuming they're okay.

Please have it say either "unless all the following are true", or have a trailing ", and" on every non-final bullet.

The spec doesn't have a lot of precedent for lists of conditions, but it seems to prefer (something like) the former, so that's what I did.

I put the normative-optional attribute only on the <ul>, even though technically, the normative-optional part starts before the "unless". Does anyone care?

Also, it occurred to me that ecmarkdown has its own way to signal a bulleted list (using *), but I don't think there's a way to mark such a list as normative-optional.

[syg]

lgtm

[bakkot]

It's not totally clear how you're supposed to interpret "[...] unless all of the following conditions are true" followed by a list marked normative-optional. The naive reading is that if you are choosing not to implement the normative optional behavior then the entire list is considered to be elided, in which case "all of the conditions" is vacuously true, which is not what we want.

Possibly something like "[...] unless the host is a web browser or otherwise supports #block-level-function-declarations-web-legacy-compatibility-semantics, and both of the following conditions are true:"? That's also more consistent with the pattern elsewhere, where the "is a browser" condition is the first guard.

[jmdyck]

Possibly something like "[...] unless the host is a web browser or otherwise supports #blah, and both of the following conditions are true:"?

Added a fixup to make this change.

Everywhere else, the phrase "the host is a web browser or otherwise supports blah is within the scope of a normative-optional, so I put the "unless ... are true:" part in a <span normative-optional> in addition to the <ul> being tagged as normative-optional. But I have no idea how that looks when rendered.

That's also more consistent with the pattern elsewhere, where the "is a browser" condition is the first guard.

There are 3 other places where I didn't make it the first guard:

• line 23091 under LabelledItem : FunctionDeclaration
• line 23241 under Catch : `catch` `(` CatchParameter `)` Block
• line 30038 in EvalDeclarationInstantiation

So you might want to confirm that you're okay with those.

[bakkot]

Ditto above.

[bakkot]

Fine for now but I'd like to find some way to inline these as well in a followup. Having a runtime algorithm step say "and then when you perform this other step, do this other thing" is crazy.

[bakkot]

Is it the case that web browsers are required to support all normative optional features? (I think yes?) If so, we should say so in the definition of "normative optional", not here.

[michaelficarra]

That may be true today, but I don't know if we want it to always be true. If it's ever not true, we run the risk of forgetting to update that text. I think the mention here is fine.

[bakkot]

I would prefer to optimize for clarity of the existing document over guarding against future editorial mistakes.

[michaelficarra]

I think it is conceptually more appropriate here as well. I don't want anyone learning "normative optional" means "available in web browsers" when that may not be true in the future, even if it is editorially corrected.

[bakkot]

My concern is that I don't want people to have to find this sentence in the annex in order to know that these features are in fact required. The fact that it's currently easy to miss that Annex B imposes additional requirements on web browsers (and therefore any engine which wants to run code written for web browsers) is the whole reason we're doing this in the first place.

I really don't think we should be worrying about being defensive against a future world where we have additional normative optional features which are optional for browsers.