Add syntax highlighting for .tscn/.tres/.godot files

[Calinou]

This is low-priority, but it'd be nice to have a dedicated highlighter for Godot scenes and resources. The automatically-detected "Properties" syntax highlighting does its job decently, but a specially tailored highlighter could do better. For instance, it could handle slashes in property names and highlight built-in Godot types.

See also #70 which is about .shader files.

[DaelonSuzuka]

I've always wanted to write a TextMate grammar from scratch. This one sounds fun.

The shader language is probably similar enough to use an existing grammar with minimal problems, but autocompletion or inline documention might need a language server or something.

[Calinou]

The shader language is probably similar enough to use an existing grammar with minimal problems, but autocompletion or inline documention will need a language server or something.

There is no LSP for the Godot shader language yet, although the list of built-in functions is known in advance (and is similar to GLSL). Using an existing GLSL grammar as a base should work well 🙂

For .tscn, I would recommend using a TOML or INI grammar as a base.

[DaelonSuzuka]

I put together a grammar that works on .tscn, .tres, .import, .gdnlib, .gdns, and .godot files.

The colors are mostly similar to the gdscript grammar, and if anybody thinks it would look better tagged a different way I can certainly swap them around.

GDResource Grammar

Godot 4 resource files appears to work, except for missing the new type names(Vector2i, PackedStringArray, PackedInt32Array, etc). Is there a master list of the new types somewhere? If somebody could point me to it, I can add them easily.

GDResource for Godot 4

And finally, here's extremely simple GDShader support. I basically just set up gdshader as an alias to glsl.

I also configured highlighting for gdshader code embedded in a gdresource file. (The right side of the below screenshot.)

GDShader Grammar

[Calinou]

@DaelonSuzuka Looks great! Feel free to open a pull request for this 🙂

Godot 4 resource files appears to work, except for missing the new type names(Vector2i, PackedStringArray, PackedInt32Array, etc). Is there a master list of the new types somewhere? If somebody could point me to it, I can add them easily.

See the diff of godotengine/godot-docs#5605 for a list of changes. Or here: https://github.com/godotengine/godot-docs/blob/13819832265a71d6e7abc2888df13a9dc6ee6146/_extensions/gdscript.py#L295-L332

[DaelonSuzuka]

@DaelonSuzuka Looks great! Feel free to open a pull request for this 🙂

Thanks! There's still some cleaning up to do, but I should have it in by the weekend.

See the diff of <snip>

Awesome, that's exactly what I needed. I'll put these in for now, and revisit this topic later and look closer for incompatibilities.

I also plan to come back and make a real grammar for GDShaders, but I figured just deferring to glsl was better than nothing.

[DaelonSuzuka]

This issue can be closed since #342 was merged.

[AlfishSoftware]

Hi. I just noticed these changes have been published.
I have been developing a separate extension for these files and gdshader which I believe is more advanced than what is currently here. I didn't publish it, but the code is public on develop branch: https://github.com/AlfishSoftware/godot-files-vscode/tree/develop
@DaelonSuzuka @Calinou You may be interested in checking it.

It has, IMO, better syntax for .tscn/.tres/etc files.
It also supports outline, hover (preload code and image/font previews) and go to definition of resource paths, which can be really useful. Please check it out.
I also added syntax for gdshader with some things I inferred from docs.

All the code is under 0BSD so feel free to use whatever you want.

In case you want to test it, make sure to rename the extension folder, so VSCode prioritizes it in gdshader language id clash (it uses alphabetic order for extension conflicts), e.g.: z-alfish.godot-files.

[DaelonSuzuka]

@AlfishSoftware Whoa, that's fantastic. Some of these features were on the roadmap already, so I massively appreciate your willingness to share.

It also supports outline, hover (preload code and image/font previews) and go to definition of resource paths, which can be really useful. Please check it out.

I had already (barely) started working on some scene/resource analysis tools, and after skimming your extension I think this will be a huge springboard. I'll definitely be in touch when I get deeper into that.

[AlfishSoftware]

I need to ask something (since I don't know of any official spec for tres|tscn|etc syntax).

In the .tscn file (at least in Godot 4) I noticed that there's a &"..." syntax. What is this?
How is it different from regular strings "..." and why is it needed?
Does it use different escaping rules (like @"..." or $"..." in C#)?
Or is it to represent a different type of value (not a string)?
Are there other similar prefixes or alternative strings?

It appeared when I made tests with VisualScript, e.g.:

[sub_resource type="VisualScriptPropertyGet" id="VisualScriptPropertyGet_58mo7"]
property = &"text"

[sub_resource type="VisualScript" id="VisualScript_urosk"]
resource_name = "MyScript"
data = {
"base_type": &"Node",

---

EDIT: Ah I see now that it's a interned StringName value from Godot 4. I'll assume it uses the same escaping rules, and that there's no other alternative string syntax (since NodePath still uses function-like syntax); please correct me if I'm wrong.

[Calinou]

There's a specification for TSCN/TRES but it's not updated for 4.0 yet: https://docs.godotengine.org/en/stable/development/file_formats/tscn.html

EDIT: Ah I see now that it's a interned StringName value from Godot 4. I'll assume it uses the same escaping rules, and that there's no other alternative string syntax (since NodePath still uses function-like syntax); please correct me if I'm wrong.

This is correct, however, I think using StringName("something") is allowed.

[AlfishSoftware]

My implementation is now released and I've published it on OpenVSX.

I have made major improvements on how inline code (GDScript and GDShader) is tokenized inside .tscn/.tres files.
Now all code should be unable to break the enclosing grammar.
It handles comments specially so things like //" /*" #" can't eat the string terminator quote.
It also has special treatment for strings, so even escape chars within GDShader strings within .tscn/.tres strings should be working now, and colorized properly. E.g.: script/source = "var s = \"Hello \\\"World\\\"!\\n\";"

Screenshot in Godot 4 format: (note the escaped escape sequences at the end)

Screenshot in Godot 3 format: (font previews are experimental)

[DaelonSuzuka]

I was able to spend some time exploring your provider class last month. I'm still learning how to read JS/TS(especially other people's JS/TS), but I learned a TON really fast. Thank you again for sharing your work so freely!

The image preview in the hover looks really good, I'll definitely have to update my local copy.

I added some extra hovers and implemented the DocumentLinkProvider interface for gdresource and gdscript files:

hovers and links example

Code_KKCiRTkaTj.mp4

I also figured out how to find project.godot even if it's not at the workspace root. This isn't done, but it works for my use case (project.godot one level deep in a subfolder). This solution breaks if there are multiple godot projects in one workspace, which is something I see recommended for developing multiplayer games.

code

async function resPathOfDocument(document: vscode.TextDocument) {
	const files = await vscode.workspace.findFiles("**/project.godot");
	if (!files) {
		return document.uri.path.replace(/^(?:.*\/)+/, "");
	}

	const project_dir = files[0].fsPath.replace("project.godot", "");

	const relative = document.uri.path.substring(project_dir.length); // extremely lazy relative path calculation, very bad
	return "res:/" + relative;
}

async function resPathToUri(resPath: string, document: vscode.TextDocument) {
	const files = await vscode.workspace.findFiles("**/project.godot");
	if (!files) {
		return resPath;
	}
	const project_dir = files[0].fsPath.replace("project.godot", "");
	return Uri.joinPath(Uri.file(project_dir), resPath.substring(6));
}

[AlfishSoftware]

I'm still learning how to read JS/TS(especially other people's JS/TS)

I was doing it mostly for my own use at first (wasn't even sure if I would end up publishing it) so I didn't bother a lot with adding comments to the code. I'll keep that in mind.

That looks cool!

how to find project.godot even if it's not at the workspace root [...] This solution breaks if there are multiple godot projects in one workspace

Yeah, I went with lazy implementation for now, but the proper way to do it is simple:

• get current document path; start with its containing folder path
• on a loop, look for a project.godot every time, going upwards (removing a path segment)
• stop on first path that contains the file; if none, then treat document as outside of any project (no res:// path can be resolved, only relative paths)
• no need to consider workspace at all, at least not for local file: URIs; not sure about others schemes, like on web-based IDE (ftp? http?); it may be prudent to not go upwards further than workspace path (this wouldn't work for subprojects)

I might work on that next, since there's still a few other things that also need fixing (like font previews)

EDIT: v0.0.2 of my implementation now has proper project detection that works regardless of open workspace or folder

[atirut-w]

Shouldn't this be closed by now? Since it seems to be implemented.

[DaelonSuzuka]

Probably, yeah.