Add language server

[ruediste]

Hi

I started implementing a language server based on langium. It already works pretty nice (modules, functions, variables, renames,...), the main missing part is handling of the builtin functions and modules (and named parameters). I'll work on this next.

This certainly isn't ready for merging, but I'd like to start getting feedback early.

Regards

[Antyos]

Thanks for the PR! Sorry it has taken me so long to get back to you on this--I have been increidbly busy with classes (I am graduating in less than a week!). It looks like you have done a very thorough job so far. I really appreciate the descriptions you added for the built-in function/module.

@Tomatower began working on a language server with #17, but we seemed to run into a bit of a roadblock while the language server API is integrated into OpenSCAD itself. One of my goals with this extension is to make it compatible for VS Code web, so having the language server running independently of OpenSCAD might not be a bad idea. I would like @Tomatower's opinion on this though.

One other thing of note--I started working on the antyos/web-extension branch a while back, but got stuck when it came to structuring code and implementing unit tests. I have been very busy the last few months with classes, so the branch has been a bit neglected, but I am planning to start working on it again once I have some more time in June.

In the Testing Extensions section of the VS Code API, they recommend using Mocha. There are a few unit tests I started writing on my other branch, but anything on that branch is subject to change until it is merged into master. I don't have much experience with JavaScript/TypeScript outside of this project, so if you think it is better to use Jest, then we can talk about that. It may also be worth opening a PR for the web extension to get some discussion going about how best to organize the code.

[Antyos]

Also, have you taken a look at dzhu/openscad-language-server?

[ruediste]

Thanks for having a look. Good luck with your graduation!

The whole idea of going this way was to leverage the huge amount of work and know-how from the langium project.

Regarding testing: I haven't done any ts testing before. I just looked at popular testing frameworks and more or less tossed a coin between mocha and jest. From what I've read mocha is more flexible than jest, which also means more work to set up and configure. If someone else does the mocha setup, I'm happy to use it.

Regarding the status of this PR: I'll start using it now for my OpenSCAD modelling to do some more testing. I don't see any big missing things right now.

One thing though: There is a performance issue in langium (something is not cached which is recomputed thousands of times, easy three liner to fix). I've opened a PR there. Right now opening a file with 100loc takes too long. For testing you can use the PR version of langium (npm link should work). I'd suggest waiting until the langium team realeases a bugfix version.

[msujew]

Hi everyone, just giving my 2 cents on the discussion as a Langium maintainer:

If you are trying to make the extension web compatible, you likely cannot use a language server as they are designed to run in a separate process in a backend. This limitation makes dzhu/openscad-language-server kind of difficult to use in a web-only context (perhaps it's possible to compile it to WASM?).

Even Langium isn't directly built for it, but it is frontend-capable. We are already using it in a closed-source frontend application, but it requires the editor (or the vscode APIs) to wire directly into the LSP services offered by Langium. It needs a bit of glue code and transformations, as the types expected by vscode/monaco aren't the same as in the LSP, but it works really well.

[Antyos]

It looks like langium/langium#497 has been merged, so are we theoretically able to merge this into master? I will be going out of town for a week and a half where I won't have access to a computer, so I may hold off on publishing a release until I get back towards the end of the month.

As for my plans to make this a web extension, as of now, I do not plan to support all the features of the desktop version. At the very least, I want to get syntax highlighting working on the web, but I do not know if I will be able to launch OpenSCAD instances to preview files any time soon.

For now, we may decide that the language server is desktop-only. There are language server extensions that work as web extensions such as Pylance, right? Can you offer any more guidance, @msujew?

[msujew]

At the very least, I want to get syntax highlighting working on the web

That shouldn't be much of an issue, since syntax highlighting (not to be mixed up with semantic highlighting) is supported without any code, but by using the vscode textmate grammar integration.

There are language server extensions that work as web extensions such as Pylance, right?

Actually, I have to revise my previous statement, since we were mostly working with standalone Monaco and not vscode-web in the context of a browser based Langium DSL. You can actually create a browser based language client which runs a language server in a worker thread. This however, limits it to using JS/TS implemented language servers (although Pylance is a Python plugin, it's implemented in TypeScript, see Pyright) or some kind of WASM interface.

In that sense, having actual vscode-web support is a matter of duplicating the existing main entry point and using vscode-language-server/browser instead of vscode-language-server/node and the correct file system access (which isn't file based in that case).

[ruediste]

Regarding the merge: the langium team has not yet released a bugfix version, so creating a performant build would require doing the packaging against the latest langium head...

[Antyos]

I forgot to come back to this--are we good to merge?

[ruediste]

I just checked. A new version of langium is out. I'll have to update an do some testing...

[msujew]

By the way, again an update regarding web extension support for Langium. If you haven't seen it yet, Langium is now (as of our next version) fully web compatible. You can try it out with our showcase.

[ruediste]

I updated langium to 0.4.0 and gave it a try. Unfortunately I immediately ran into NPEs deep in langium. @msujew: Would it be possible to have a look at this? Sample input:

x=function (a) a*a;
echo(x(3));