Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

request for feedback: editorial change to restructure SDOs #1950

Closed
michaelficarra opened this issue Apr 15, 2020 · 28 comments · Fixed by #2271
Closed

request for feedback: editorial change to restructure SDOs #1950

michaelficarra opened this issue Apr 15, 2020 · 28 comments · Fixed by #2271

Comments

@michaelficarra
Copy link
Member

michaelficarra commented Apr 15, 2020

In the most recent editor call, we were discussing general solutions to linking to SDOs such as those described in #1927 and #1821. I proposed a solution of restructuring the specification to group the definitions of each SDO together, instead of grouping all SDO definitions for each production together. This would allow us to link to the one place where an SDO is defined, and we believe would be generally more useful for readers. But we wanted to allow time for readers to give us feedback before this fairly significant restructuring of ECMA-262.

edit: WIP concrete proposal, will edit as conversation below develops:

  • move all SDO definitions into a new section
    • sort them alphabetically
    • early errors remain inline where they are
    • Evaluation gets its own top-level section
  • replace Annex A (Grammar Summary) with an Early Errors Summary
@bakkot
Copy link
Contributor

bakkot commented Apr 15, 2020

I think restructuring the specification so that all the definition for each operation is in a single place, rather than spread across the various places each relevant syntactic form is defined, would be an improvement.

@devsnek
Copy link
Member

devsnek commented Apr 15, 2020

yes yes so much yes

@syg
Copy link
Contributor

syg commented Apr 15, 2020

As an editor and a reader, definitely in favor as well.

@jmdyck
Copy link
Collaborator

jmdyck commented Apr 16, 2020

In addition to providing a singular link target for a given SDO, it would also provide a place where one could describe the general semantics of the SDO (if that were deemed useful).

Would you put all the SDOs in one place, or locate each at some appropriate spot in the spec, or a combination of the two approaches?

Early Errors are syntax-directed rules; but I gather they'd stay where they are?

@michaelficarra
Copy link
Member Author

@jmdyck

Would you put all the SDOs in one place, or locate each at some appropriate spot in the spec, or a combination of the two approaches?

I was envisioning they'd have their own section (maybe Evaluation is split out?). A nice alphabetical listing of them.

Early Errors are syntax-directed rules; but I gather they'd stay where they are?

We discussed this during the call, actually. It'd probably be best to leave them where they are. We can also have a new annex that has a copy of all of them in one place.

@michaelficarra
Copy link
Member Author

Now that I think about it, there won't be much of a reason for Annex A anymore, will there?

@jmdyck
Copy link
Collaborator

jmdyck commented Apr 16, 2020

Are you thinking that, once all the SDOs have been pulled out and collected in one section, what remains in the "body" of the spec would present the grammar in a 'concentrated' enough form that we wouldn't need Annex A?

Well, certainly, clauses 11 through 15 would be a lot slimmer, but they wouldn't just be grammar. There'd also be:

  • prose (sometimes extensive) that accompanies Syntax,
  • Early Errors,
  • some abstract operations,
  • other odds and ends (especially re Names + Keywords, ASI, Iteration, Scripts, Modules).

Or are you thinking that some of that stuff would also migrate elsewhere, leaving basically just Syntax + Early Errors?

@michaelficarra
Copy link
Member Author

I'm thinking that what remains will be fairly compact, yes. Once we've extracted the SDOs, we could get a better idea of whether it would make sense to move the AOs defined in those sections as well. But if we did do that, I think it should be a separate step.

@jmdyck
Copy link
Collaborator

jmdyck commented Apr 19, 2020

Moving that much text has the potential to create lots of merge conflicts for in-flight PRs (and pre-PR branches). Would you try to minimize that by resolving such PRs before extracting the SDOs?

@bakkot
Copy link
Contributor

bakkot commented Jun 1, 2020

@waldemarhorwat points out that it would be useful for there to be links from each production to all of the SDOs which are defined for it. We should do that too.

It should be possible to make that happen automatically from ecmarkup with a little work, though the definitions which are omitted because of the chain production rule would probably not get links without doing substantially more work. Since those aren't mentioned next to the productions currently, that omission will at least not make the state of affairs worse.

@bakkot bakkot reopened this Jun 1, 2020
@bakkot
Copy link
Contributor

bakkot commented Aug 16, 2020

Fun fact: according to Allen's JavaScript: The First 20 Years, this style is borrowed from the LISP 1.5 Programmer's Manual, specifically (the paper speculates) from the description of the PROG feature:

He recalled [Welland et al. 2018, at +10:10] that the LISP 1.5 Programmer’s Manual [McCarthy and Levin 1965] had described the semantics of the Lisp interpreter using a style where a syntactic form was immediately followed by a precise description of how that syntax should be evaluated. In some cases, the semantics were presented using pseudocode.33 Welland decided to use a similar style of pseudocode with numbered steps to describe the evaluation semantics of JavaScript

(The spec with this style was competing with Brendan Eich's spec at the initial TC39 meeting, and was chosen over it on the basis that it was written in Word and Ecma liked Word.)

@jmdyck
Copy link
Collaborator

jmdyck commented Oct 15, 2020

The spec has two distinct activities that are both referred to as evaluate/evaluating/evaluation:

  • execute an ES expression, resulting in an ES lanuguage value; and
  • compile a RegExp, resulting in an Abstract Closure.

Currently, they're in different parts of the spec, so the overlap in terminology is not a big deal. But if the restructuring proposed here brought them together as a single SDO, that might be confusing. So you might want to rename one of them first.

@jmdyck
Copy link
Collaborator

jmdyck commented Oct 21, 2020

@waldemarhorwat points out that it would be useful for there to be links from each production to all of the SDOs which are defined for it. We should do that too.

It should be possible to make that happen automatically from ecmarkup with a little work,

Be careful that the links don't clutter up the grammar. (E.g., have them appear only on request.) I think the "worst case" is LabelledStatement, for which 14 different SDOs have an explicit definition:

  • 13.13.2 ContainsDuplicateLabels
  • 13.13.3 ContainsUndefinedBreakTarget
  • 13.13.4 ContainsUndefinedContinueTarget
  • 13.13.6 LexicallyDeclaredNames
  • 13.13.7 LexicallyScopedDeclarations
  • 13.13.8 TopLevelLexicallyDeclaredNames
  • 13.13.9 TopLevelLexicallyScopedDeclarations
  • 13.13.10 TopLevelVarDeclaredNames
  • 13.13.11 TopLevelVarScopedDeclarations
  • 13.13.12 VarDeclaredNames
  • 13.13.13 VarScopedDeclarations
  • 13.13.14 LabelledEvaluation
  • 13.13.15 Evaluation
  • 14.9.2.1 HasCallInTailPosition

@bakkot
Copy link
Contributor

bakkot commented Oct 21, 2020

@jmdyck what I was thinking was that those sections would be collectively replaced by a list of the SDOs: so instead of having 13.13.2-15 (or 14.9.2.1), there would be a single 13.13.2 which contained a 14-item list of SDO names, with those linking to the relevant part of the named SDO. I hope that wouldn't be too cluttered, but we'll have to experiment.

(I was a little imprecise above: I didn't mean to say the names would be following each production but rather each set of productions, just the SDOs themselves do now.)

@jmdyck
Copy link
Collaborator

jmdyck commented Oct 21, 2020

Okay, that would leave the grammar easy to read.

When you say "with those linking to the relevant part of the named SDO", do you mean that each SDO-clause would be divided into parts? (E.g., the clause dedicated to ContainsDuplicateLabels might have one part that's basically the current 13.13.2, and another that's 13.6.2, and so on?)

@bakkot
Copy link
Contributor

bakkot commented Oct 21, 2020

What I'm currently envisioning is that it would not be divided in the sense of having a visual separation between parts, but rather it would have ids on productions such that it was possible to link to a specific production, so that the aforementioned links could take you directly to the relevant portion of the SDO.

@jmdyck
Copy link
Collaborator

jmdyck commented Oct 21, 2020

So, e.g., would the new 13.13.2 13.2.2 be organized like this?

  • ContainsDuplicateLabels

    • Block : { } [link to ContainsDuplicateLabels for that prodn]
    • StatementList : StatementListStatementListItem [link to ContainsDuplicateLabels for that prodn]
    • StatementListItem : Declaration [link to ContainsDuplicateLabels that prodn]
  • ContainsUndefinedBreakTarget

    • Block : { } [link to ContainsUndefinedBreakTarget for that prodn]

etc

or like this?

  • Block : { }

    • ContainsDuplicateLabels [link to ContainsDuplicateLabels for that prodn]
    • ContainsUndefinedBreakTarget [link to ContainsUndefinedBreakTarget for that prodn]
    • ....
    • Evaluation [link to Evaluation for that prodn]
  • StatementList : StatementListStatementListItem

    • ContainsDuplicateLabels [link to ContainsDuplicateLabels for that prodn]
    • ...
    • Evaluation [link to Evaluation for that prodn]

etc

The first sounds like what you were saying (a single clause with a 14-item list of SDO names), but the second sounds like what @waldemarhorwat was reported to suggest (links from each production to all of the SDOs which are defined for it).

I suppose it could be either, at user option, because the content is going to be generated, right?

@bakkot
Copy link
Contributor

bakkot commented Oct 21, 2020

Why would 13.13 contain a section for Block : { }? I would put that under 13.2, where the Block : { } production is defined.

As such, I was expecting 13.13 to be just

  • ContainsDuplicateLabels
  • ContainsUndefinedBreakTarget
  • ContainsUndefinedContinueTarget
  • [...]

(With those being links, of course.)

Some sections have multiple productions defined, not all of which have all the relevant SDOs defined over them, but I am hopeful that won't be too confusing.

I suppose it could be either, at user option, because the content is going to be generated, right?

It could, though I'm probably not going to build support for other modes unless they're requested or seem particularly useful. And we do need to pick a canonical one for the PDF rendering sent to ECMA.

@jmdyck
Copy link
Collaborator

jmdyck commented Oct 21, 2020

Why would 13.13 contain a section for Block : { }?

Sorry, replace "13.13" in my comment with "13.2".

As such, I was expecting 13.13 to be just

* ContainsDuplicateLabels
* ContainsUndefinedBreakTarget
* ContainsUndefinedContinueTarget
* [...]

(With those being links, of course.)

Some sections have multiple productions defined, not all of which have all the relevant SDOs defined over them, but I am hopeful that won't be too confusing.

So given that the current 13.13.2 defines ContainsDuplicateLabels for 2 different productions, the new 13.13.2's "ContainsDuplicateLabels" would be a link to (say) whichever of those productions occurs first in the ContainsDuplicateLabels clause?

@bakkot
Copy link
Contributor

bakkot commented Oct 21, 2020

the new 13.13.2's "ContainsDuplicateLabels" would be a link to (say) whichever of those productions occurs first in the ContainsDuplicateLabels clause?

Yup. But the productions would be adjacent within that clause, ordered the same way they are in the grammar, so hopefully it would be easy to find the one you were looking for.

@jmdyck
Copy link
Collaborator

jmdyck commented Oct 21, 2020

Yeah, that might work. Compared to the current rendering, it's got about the same amount of clicking-and-scrolling to get from a production to an SDO's definition for that production. It could be about optimal for the PDF rendering.

But it might seem kind of coarse-grained to someone who wanted "links from each production to all of the SDOs which are defined for it". Here's a possible enhancement: currently, when you hover over a production, you get a pop-up with "Permalink Pin References". Consider adding "SDOs", which (similar to "References") opens a pane that lists all the SDOs defined on that production, each being a link to that SDO's definition for that production. (Even more fine-grained would be to do it per right-hand-side.)

@bakkot
Copy link
Contributor

bakkot commented Oct 21, 2020

The "References" section of the popup actually already contains links to the SDOs defined over the production, going specifically to the definitions corresponding to that production. We could just keep that.

@tc39 tc39 deleted a comment from peterchykeonlinode Oct 22, 2020
@jmdyck
Copy link
Collaborator

jmdyck commented Oct 22, 2020

The "References" section of the popup actually already contains links to the SDOs defined over the production

No, I don't think it does. It looks to me like it contains links to places where the production's LHS-symbol occurs in the RHSs of other productions, or in algorithms, or in prose. Some of which might be in SDO definitions, but those would not generally be SDO definitions for the production in question.

E.g., consider the IfStatement production. The References pane has 3 references, none of which are to SDO definitions. Yet 7 SDOs have definitions for IfStatement. Or look at the Block production. In 13.2.*, there are 10 SDOs defined on that production, but the "References" pane doesn't link to any of those definitions. It's the same with other productions I've tried.

(One situation where it might look otherwise is when a production's LHS-symbol also appears on one or more of its RHSs, because then when the References pane links to RHS occurrences of that symbol, some of them might happen to be from the production in question.)

@bakkot
Copy link
Contributor

bakkot commented Dec 11, 2020

We should be sure to do this for Contains as well (cf #2251).

@jmdyck
Copy link
Collaborator

jmdyck commented Dec 13, 2020

The SDO definitions in the RegExp clause are the only ones in the 'standard library' part of the spec. Are you thinking that they would be merged in with all the other SDOs, or would they undergo their own re-org, but stay within the RegExp clause?

@bakkot
Copy link
Contributor

bakkot commented Dec 13, 2020

That's a good question. I'm inclined to leave them where they are - they are defined over a different grammar, after all. They also have the property that their definitions are not spread out: CapturingGroupNumber, for example, is only defined over two productions, and those definitions appear in the same clause. So there's much less reason to reorganize. I'm not sure they need to be reorganized at all.

@bakkot
Copy link
Contributor

bakkot commented Dec 24, 2020

Looking into this more concretely:

There's a few special cases. HasCallInTailPosition is already organized essentially as proposed here. Evaluation and Early Errors are defined for most nodes, and hence probably shouldn't be reorganized.

Of the remainder, 46 have definitions which are already contained in a single clause:

  • NumericValue
  • BodyText
  • FlagText
  • CoveredParenthesizedExpression
  • ArrayAccumulation
  • IsComputedPropertyKey
  • PropertyNameList
  • TemplateStrings
  • SubstitutionEvaluation
  • CoveredCallExpression
  • ChainEvaluation
  • DestructuringAssignmentEvaluation
  • PropertyDestructuringAssignmentEvaluation
  • RestDestructuringAssignmentEvaluation
  • IteratorDestructuringAssignmentEvaluation
  • KeyedDestructuringAssignmentEvaluation
  • DeclarationPart
  • PropertyBindingInitialization
  • RestBindingInitialization
  • KeyedBindingInitialization
  • BindingInstantiation
  • CaseBlockEvaluation
  • CatchClauseEvaluation
  • CoveredFormalsList
  • SpecialMethod
  • DefineMethod
  • ClassElementKind
  • ConstructorMethod
  • IsStatic
  • NonConstructorMethodDefinitions
  • PrototypePropertyNameList
  • ClassDefinitionEvaluation
  • BindingClassDeclarationEvaluation
  • CoveredAsyncArrowHead
  • IsStrict
  • ImportEntriesForModule
  • ExportEntriesForModule
  • ReferencedBindings
  • CapturingGroupNumber
  • IsCharacterClass
  • CharacterValue
  • SourceText
  • StringValue
  • CharacterEscape
  • DecimalEscape
  • ClassEscape

And only 41 have definitions which are split across multiple clauses:

  • StringValue
  • BoundNames
  • AssignmentTargetType
  • BindingInitialization
  • HasName
  • IsFunctionDefinition
  • IsIdentifierRef
  • ComputedPropertyContains
  • Contains
  • PropName
  • PropertyDefinitionEvaluation
  • ArgumentListEvaluation
  • NamedEvaluation
  • IsDestructuring
  • ContainsDuplicateLabels
  • ContainsUndefinedBreakTarget
  • ContainsUndefinedContinueTarget
  • VarDeclaredNames
  • VarScopedDeclarations
  • LabelledEvaluation
  • LexicallyDeclaredNames
  • LexicallyScopedDeclarations
  • TopLevelLexicallyDeclaredNames
  • TopLevelLexicallyScopedDeclarations
  • TopLevelVarDeclaredNames
  • TopLevelVarScopedDeclarations
  • IsConstantDeclaration
  • ContainsExpression
  • HasInitializer
  • IsSimpleParameterList
  • IteratorBindingInitialization
  • ContainsUseStrict
  • ExpectedArgumentCount
  • EvaluateBody
  • InstantiateFunctionObject
  • HasDirectSuper
  • ExportedBindings
  • ExportedNames
  • ExportEntries
  • ImportEntries
  • ModuleRequests

The original proposal was to put all SDOs into a single clause, but I'm no longer sure that makes sense. I'm inclined to only reorganize this last group of 41, and to try to put each in the location where it makes the most sense, just as other abstract operations are spread across the specification wherever they make the most sense.

(In several cases there's only a single callsite, so we can just put the definition after the algorithm which calls it.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants