missing architecture docs

[OmarTawfik]

A new section under https://nomicfoundation.github.io/slang/user-guide/architecture/syntax

Document: CST

• How it is a complete tree: every byte is a node.
• What are rule nodes and what are token nodes.
• How to navigate their children, etc…
• How you can parse any ProductionKind, not just SourceUnit.

Document: Grammar

• Grammar pages and navigating rules.
• Mapping ebnf to rule and token kinds.
• Inlined nodes.
• Different EBNF operators.
• How precedence and associativity are specified.

Document: Trivia

• Leading vs trailing, and how they are grouped.
• Why most tools will traverse the tree ignoring it
• How an IDE or a formatter can use it to handle comments.
• How to extract NatSpec and parse it externally for now.
  • Can users use available jsdoc parsers to post-process it?
  • Add note that it is on our roadmap to provide that API.

Document: Visitors

• How each node stores lengths or text and how ranges are calculated during visits.
• Different use cases + code examples.
  • Add note that it is on our roadmap to additionally provide cursors + rewriters.

Document: AST

• Perhaps just a single paragraph about how we plan on providing a strong types for nodes.
• Only built for successfully parsed CSTs.
• How can they abstract over version differences.
• Providing the same visitors + cursors + rewriters experience.