(draft) Need for token typing. The key/value structure of JSON complicates things.

[chazzmoney]

People want to be able to do creative, useful things with SD. While we provide a flexible mechanism of JS to define files, people tend to like the simplicity, usefulness and familiarity of JSON (for a variety of reasons) - which was why it was the original (and default) method to define dictionaries. However, by its very nature JSON does not provide typing information regarding the key/value pair, making it hard to distinguish the use of a key. We currently use the CTI structure as a substitute mechanism to perform logic that should be done via type (e.g. category is the type)

Specifically, within the JSON we distinguish a token by the "value" keyword. We have a few others (e.g. "comment", "deprecated") to define additional metadata. To a certain extent this is strange (hacking JSON to meet our needs), and leads to the obvious desire to remove the value keyword that dominates the files (and makes them busy).

Additionally, when someone wants to do something beyond the standard transform and output, they often see the obvious solution of just adding a key or two of metadata in the JSON to meet their needs within their custom transformer/formatter. However, this is problematic for a few reasons:

• conflicts between different uses of the same metadata key may occur
• expectations on the inheritabliity of metadata differs from use to use
• metadata comes in a variety of types: grouping/organization, logical/flags, etc
• metadata is intended for a variety of uses: transform modification, format logic, additional format output, etc

This structural incompatibility between JSON and Style Dictionaries is the root cause of a variety of issues: (TODO: list issues)

• [scss/variables] Nested token name is omitted
  [scss/variables] Nested token name is omitted #366
• Define colors without "color" key first? Using references to colors #659

TypeScript and XML/YAML both could be potential solutions to this problem but promoting either of them as a new default format for SD would be difficult and challenging and likely end up forking the user base. Suggesting that JS/TS/XML/YAML and custom parsers / custom formats already solve the problem is tone deaf to the complex needs of our users. So what would be the best solution?