Language Grammar Highlighting Messed Up? (Latest Insiders)

[bradennapier]

Version: 1.42.0-insider
Commit: d72b2d3
Date: 2020-01-20T05:27:44.927Z
Electron: 7.1.7
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Darwin x64 19.3.0

Steps to Reproduce:

1. Open Javascript File
2. Use code shown

I believe this has broken languages for me. I am using "Babel Javascript" and noticed with the most recent update that my color themes are screwed up.

• It happens with the built in "Javascript" grammar as well as the "Babel Javascript" plugin
• This happens with extensions disabled.
• Interestingly the highlighting is actually correct until about 3-4 seconds after a window loads.
• This happens when using default color themes as well
• There does not appear to be any difference when I inspect the tokens - they refer to the same types.
• Not sure if this is in latest insiders build, but it is main thing i see that could cause potentially that is recent? 852f493
• Another potential culprit? fd13e44
• Also noticed TypeScript brace matching is incorrect in ${} strings #88075

Recent Insiders Build:

Regular VSCode & Previous Insiders Builds:

const day = require('dayjs');
const SequelizeMysqlConnectionManager = require('sequelize/lib/dialects/mysql/connection-manager');
const abi = require('./abi');

const { env } = process;

Happen without Extensions?

Yes

[IllusionMH]

Probably same root cause as #88911

[mjbvz]

Likely caused by semantic highlighting. Try setting "editor.semanticHighlighting.enabled": false to disable it

[aeschli]

Not a bug but semantic highlighting that shows the actual type of the import based on the theme
yellow for functions, blue for variables.
If you don't like that information you can turn semantic highlighting off by
"editor.semanticHighlighting.enabled": false

[mjbvz]

@aeschli This is a major visual change that is going to cause a lot of issue reports against JS/TS. Are we shipping 1.42 with the semantic highlighting enabled by default? Anything we can do to reduce the impact?

[IllusionMH]

@aeschli it's understandable how it works, but it brings few problems to JSX:

1. It creates pale pale tree without almost no highlights if you use const Comp = () =>(<div>Hi</div>) because there is no color diff between <div>, <Comp /> and attribute name
   
2. It creates visual difference in tree between different implementation, but inside JSX thee all are used in the same way and difference should be between components and regular HTML tags

class CompCls extends React.Component{}

function CompFun() {}

const CompArr = () => {}
//or
const CompRef =  React.forwardRef(CompCls)

IMHO In this case semantic meanings shouldn't rely on original types, but differentiate between (custom) components, regular tags, and attributes. This is not in place currently.

---

So there is need to have extra info about cases when function/class/variable/property are used in tagName of JSX element.
And easy way to restore previous highlighs: green(-ish) for (custom) components, dark blue for tags, light blue for attributes that make sense in that place.

[aeschli]

We can turn it off by default for stable, but we should work through the various issues.

• imports (see example above): For me there's a value-add with semantic coloring. Of course there are still issues, such as what to do if a identifier is both a class and a variable.
• JSX tags: I'll look at that. Obviously we don't want everything to become blue.

[bradennapier]

There is more than just imports affected here. @aeschli maybe i dont understand specifically but for example a while statement is now yellow. sometimes my highlgithing is half words gray and half blue, will try to get screenshot when i can.

Other times imports have 3 different colorings - gray, yellow, blue based on my theme. How many colors are there now? Is there info on what this is doing exactly?

This seems to make the code really hard to read. I know i can turn it off, but i assume it is an ideal feature for other reasons and want to understand it before i just turn it off.

A bit confused why for statement is purple

and a while is yellow

I am guessing that it is trying to make functions one color and variables another, etc which is cool - I like that a lot. But seems inconsistent?

Like this chain state.db.updatedUsers changes the color after the first property is accessed

It very well could just be that my color theme isn't really setup well for the semantic highlighting ofc.

needless to say can definitely see this causing a huge stir if it is default in a release people randomly get hit with. do love the idea of syntax highlighting being smart though - that would be awesome if it produced code that is equally or even more easy to read and grok!

[aeschli]

@bradennapier Thanks for the help to find all the issues.

• Changing JSX tags was not intended. Fixed. Semantic highlighting changes colors of JSX tags #88911

Semantic highlighting only looks at identifiers and resolves them to find out to what symbol they resolve.
You can use the Developer: Inspect Editor Tokens and Scopes hover to see what semantic type/modifier was given to an identifier.
We have a map that assigns each semantic token types to one or more textmate scopes. We query the theme for these scopes to evaluate the color to use.

Keywords are not touched. I tried to reproduce your example but I don't see any change for while. I'd like to understand why CONFIG.updateInterval gets different coloring. All identifiers should be classified as variable and get the color used for the variable color (scopes ['variable'], ['entity.name.variable'])

• Please make sure you are on the latest i-build
• What theme are you using?
• Can you attach a small code sample (not just the screenshot)?
• What does the Inspect Editor Tokens and Scopes hover show as semantic type and modifier.