Semantic Tokens for TypeScript Template Literal Strings

[benrbray]

(moved from typescript-language-server/typescript-language-server#901)

Problem

In VS Code, semantic token theme colors override textmate token colors (even when they are user defined). This usually makes sense, as the semantic tokens are intended to provide more specific information about tokens than what is available from syntax alone.

However, based on the output of VS Code's Developer: Inspect Editor Tokens and Scopes command, it appears that typescript-language-server labels all template strings with the generic string scope. As a result, template strings will not receive syntax highlighting when semantic tokens are enabled, degrading the user experience writing TypeScript when semantic highlighting is enabled.

As you can see from the image below, the textmate grammar knows that animal is a variable inside of a template string, but the semantic token string is taking precedence.

Possible Solutions

• The best would be to add more granular semantic tokens for template literal strings.
• Another possible solution would to allow more fine-grained control over the interaction between semantic highlighting and textmate highlighting. At the moment, semantic highlights always get priority.

Related Issues

• Add Semantic Tokens for Template Literal Strings typescript-language-server/typescript-language-server#901
• [semantic] proposals for new standard semantic token types  vscode#97063

[benrbray]

I've also noticed that the following keywords are not given their own semantic token type (instead, they are treated as variable) only:

• null
• true
• false