Slow server startup and commands

[PyvesB]

Hello @castwide,

First of all thanks for your continued work on this project, it's great to see that Solargraph keeps on improving! 👍

I guess this issue is in essence quite similar to #164. The server seems to take a long time to start up, even for a project with a single Ruby file.

Using Solargraph within Eclipse and observing the requests and responses between the client and the server, it regularly takes over 10 seconds for Solargraph to respond to the first initialize request. It seems equally slow from the command line, when running solargraph stdio I only see the Solargraph is listening on stdio PID message after a few seconds and if my understanding is correct the server is not yet initialised at that point. These poor timings can partly be explained by the fact that my machine has fairly low specifications by today's standards (AMD Quad-Core 2.1 GHz, 8 GB memory, Windows 10, Ruby 2.6.1). However, other language servers take much less time with projects of similar size, the Javascript, Typescript and XML servers available through other Eclipse plugins all take about a second to respond to the initialize request. Code completion and intellisense are provided almost instantly.

I guess it's somewhat unfair to compare different servers built using different technologies and targetting different programming language, so I've also looked at performance under a different angle. I noticed that all Solargraph commands generally seemed quite slow in a terminal, so I compared the response times of the --help and --version commands of common gems, as they pretty much all implement these. This time round, I used a higher spec machine (modern Intel Core i7 2.4 GHz, 16GB memory, macOS Mojave, Ruby 2.6.2). I used the time command to gather the below figures, averaged on about 5 executions each:

gems	version command	help command
bundle	0.17s	0.19s
rspec	0.23s	0.20s
yard	0.18s	0.45s
pry	0.49s	0.50s
rubocop	0.74s	0.75s
solargraph	1.30s	1.39s

Solargraph is significantly slower than other gems for these two cases. I'm no expert when it comes to implementing such commands in gems, but something does not seem quite right.

Thanks for reading and hopefully these observations will be somewhat useful! Let me know if I can provide any more information. 😉

Cheers,

Pyves

[castwide]

There are two issues involved here: startup time and server initialization time.

Startup time is comparably slow in your benchmarks because the shell loads a lot of components that should not be necessary for simple commands like --help and --version. With a bit of refactoring, we can probably get execution time well below 1 second.

Server initialization is a bigger issue. One of the most expensive parts of initialization is parsing the workspace. According to my benchmarks, the parser is capable of processing ~1600 lines of code per second. For a 30KLOC workspace, parsing is responsible for about 18 seconds of the initialization time.

We might be able to reduce that time with an implementation that uses Ripper instead of whitequark/parser. According to my preliminary benchmarks of Ripper::SexpBuilder, it's capable of processing ~87,000 lines per second, more than 50x faster than the current implementation.

Switching from Parser to Ripper would not be a trivial change. It involves one or both of 1) extending SexpBuilder to emulate the AST generated by Parser, or 2) modifying components like Source::Chain and SourceMap::NodeProcessor to handle the new AST architecture. There's also some risk that the additional functionality would significantly increase the processing time, although I suspect it would still be faster than Parser.

Given the amount of work involved, I can't estimate an ETA on progress yet. Nevertheless, improving performance is an ongoing concern, so any suggestions and feedback are appreciated.

(Benchmarks were performed on a Windows 10 PC with 32 GB RAM and an 8-core 4.0 GHz CPU.)

[castwide]

A new commit to master refactors dependency loading to improve startup time. Using time on a CentOS desktop, running --version and --help went from ~1.7s to ~0.4s.

It looks like I'll be switching gears on the server initialization issue. I tried a prototype of the parser that uses Ripper. The resulting AST does not include all the necessary data that Parser provides. I tried extending Ripper::SexpBuilder to gather the missing data, but as I feared, the added functionality drastically increased processing time. It no longer seems practical to build a custom AST parser, since the result would be less robust than Parser and would not perform significantly better (if not worse).

There's still another option, though. I've been experimenting with RubyVM::AbstractSyntaxTree and the results are much more promising. Its output has a different architecture from Parser, but it provides all the same data and appears to perform even faster than Ripper. It'll still require a major redesign. It also requires Ruby 2.6+, so I'll probably have to add a mechanism to fall back to the legacy parser for earlier versions.

[castwide]

Version 0.38.0 includes the improvement to base startup speeds. The new RubyVM feature to improve parsing time is still in progress.

[PyvesB]

Version 0.38.0 includes the improvement to base startup speeds. The new RubyVM feature to improve parsing time is still in progress.

Thanks, will try it out and report on any anomalies!

[castwide]

The rubyvm branch introduces the new RubyVM parser for Ruby 2.6+. The benchmarks are promising. Initialization time on the rails/rails repo in VS Code went from 57 seconds to 46. Earlier Ruby versions are also likely to see a small performance boost due to some changes to require processing in ApiMap.

The only major issues I'm seeing are with the type checker, which is still tightly coupled to the AST implementation.

[castwide]

Current efforts to enhance performance are being tracked in #268.