Clarification of terms "symbol" and "token"

[avekens]

At the beginning of section 4.2, it is clearly defined that:

• "... tokens ... are strings of non-whitespace printable characters ... separated by white space ...".
• "... tokens ... consist of keywords ... together with user-defined tokens"
• user-defined tokens are either labels or (math) symbols.

At least in section 4.4.2, however, these strict definitions seem not to be used consequently:

• the term "math token" or "math-token" is used (but not defined). Does it mean "math symbol"?
• $t is sometimes called "token", sometimes "symbol", but according to the strict definition it should be called a "keyword"

Should section 4.4.2 be revised accordingly?

[benjub]

It's true that "symbol" is not used the same way in the book. That said, since in the book's sense, a symbol is one of two kinds of token, what I wrote is correct :) Plus, this permits the distinction of the string in the mm file and the symbol displayed on the webpage. Since it's in my mathbox and I'm away from my computer I'll leave it this way for now, but I'll conform to any convention and modify it if needed. Le jeu. 6 janv. 2022 à 19:29, avekens ***@***.***> a écrit :

…

At the beginning of section 4.2, it is clearly defined that: - "... *tokens* ... are strings of non-whitespace printable characters ... separated by white space ...". - "... tokens ... consist of *keywords* ... together with *user-defined tokens*" - user-defined tokens are either *labels* or *(math) symbols*. At least in section 4.4.2, however, these strict definitions seem not to be used consequently: - the term "math token" or "math-token" is used (but not defined). Does it mean "math symbol"? - $t is sometimes called "token", sometimes "symbol", but according to the strict definition it should be called a "keyword" Should section 4.4.2 be revised accordingly? — Reply to this email directly, view it on GitHub <#231>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJZP566OLP2BZZW6TXX5W2LUUXNPXANCNFSM5LM6AVRQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[benjub]

My comment above sounds a bit off since I thought I was posting in #2422, which triggered this issue. Sorry about that. Putting the discussions there and here together, this makes more sense.
Maybe a solution is to replace "(math) symbol" by "math token" everywhere in the book ? This would permit to reserve the word "symbol" for what is display on a webpage. E.g., the $t statement would look like:

htmldef "math-token" as "gif-symbol";
althtmldef "math-token" as "unicode-symbol";
latexdef "math-token" as "latex-symbol";

But honestly, I think it can stay as is and is not a big problem. The formal grammar is given in Appendix E. For instance, it has:

essential-stmt ::= LABEL ’$e’ typecode MATH-SYMBOL* ’$.’