Library/Dependency information

[gorkem]

The LSP should define a request from the client to server to obtain the libraries/dependencies on workspace that can be used to display and browse the 3rd part dependencies of a project.

[mickaelistria]

That would be a good feature.

However, it brings some questions:

• Assuming we have a LS with 2 different and unrelated files open, and LSP doesn't really have a concept of resolving a given file to a project/dependency set. It seems to me that without introducing in LSP a structure of "projects", then the request would have to be made once for each file. LSP has the concept of workspace roots, but if you take JDT.LS as example, there is one workspace root but several dependency sets (basically one per Maven module or even per Maven source folders).
  How would you expect for example to have the Maven depedencies rendered in an IDE using JDT-LS and showing basically a filesystem tree? Dependencies would be added as children on folders and apply to the folder children until another child overrides the dependencies? If so, what would be the right request to send to LS? Ask for all known dependency sets in the workspace? Or maybe something like workpsace/resourcesWithDependencySets: void -> URI[] resources (typically resolving to Maven modules) and then workpace/resolveDependencies: URI resource -> Dependency[] ?
• What is the final use-case you have in mind? Is this a one similar to the one above -inclusion in the tree view of a project-, or mostly a way for a users to discover possible dependencies to add to the project?
• It would requires a way to browse a dependency content, but this is not necessarily a filesystem content so we couldn't rely on expected filesystem browser provided by the IDE and would have to define a browsing model for dependencies in the LSP. Do we have strong enough semantics of what are the content of a dependency to define a common data type in the LSP? Basically, imagining we are using the requests as specified as above, what sould be the content of Dependency to cover the use case well?

[keegancsmith]

There is some related discussion in #470 specifically the linked Build Server Protocol may be able to answer these questions.

At Sourcegraph we need to solve this problem around dependencies. The extensions we built on top of LSP allows us to provide cross repository find references for example. I'd be happy to discuss them further and maybe we can come up with proposals to LSP and/or BSP. It is a bit out of date, but you can see in master and PRs some of the changes we made to LSP https://github.com/sourcegraph/language-server-protocol

[mickaelistria]

we need to solve this problem around dependencies.

Can you detail what is this problem in your case. I'm not sure your need for dependencies is the same as @gorkem 's one and that the same solution can fit both.
What's the user story you want to cover?

[keegancsmith]

This sounds like what @gorkem wants?

We additionally use that data to also build indexes.

[tsmaeder]

The model we were discussing recently is more about enhancing the browsing model than anything else. Basically, the language server could add nodes to the children of a particular folder URI. In this way, the semantics of this would be completely up to the language server.

[mickaelistria]

Reminds me of Eclipse Common Navigator Framework and TreeContentProviders ;)
This indeed seems like something necessary in any IDE at some point. So it's not only dependencies but about adding treeNodes. So a request could be something like workspace/extraTreeNodes: URI|ExtraNodeUUID node -> ExtraNode[] children, with ExtraNodeUUID: {String uuid} and ExtraNode: { ExtraNodeUUID uuid; String label; String? iconData}.
Is this the kind of stuff you have in mind?

[gorkem]

@keegancsmith as @tsmaeder said we had more of a browse model on mind which may include browsing inside the dependency too. Your case probably needs more attributes than we had in mind to be useful for you. Do you have any ideas on what kind of data you need?

[tsmaeder]

@mickaelistria LSP uses URIs to identify stuff. If an implementor would like to put a UUID in the URI, that's their prerogative, but I would not mandate it in the API.

[keegancsmith]

We heavily rely on using custom URIs to achieve cross dependency browsing. Our uris are essentially git://git/clone/url?path/to/file#rev. However, in the case of dependencies you quite often don't know much about the provenance of a package, all you have is the artefact you get from the package manager. So our attributes we store/index are really as much info as we can get about a dependency, but its very much language specific.

[dbaeumer]

From my experiences these kind of information need to come from a project system which in many cases are connected to the used build system. Taking TS as an example this information needs to come from the settings in the tsconfig.json file since it determines which lib*.d.ts file to use. I like the discussion around this especially in the light of the proposed build server protocol. For the BSP we need to think about that in complex systems the LSP needs to consult the build system to for example correctly handle requests like call hierarchies / find references.

[tsmaeder]

@dbaeumer Yes, precise cross referencing needs to know the project model. So what is your suggestion?

[jvican]

For the BSP we need to think about that in complex systems the LSP needs to consult the build system to for example correctly handle requests like call hierarchies / find references.

I agree. In my experience, getting this information from tsconfig.json falls short for more complicated use cases, and I believe there is a lot of value in connecting language server/editors with build tools via BSP.

In the case of Scala, for example, our build systems are flexible and customizable, and that makes it difficult to get correct diagnostics from the language servers alone since all the compiler information and project dependencies live in the build systems.

Our goal with BSP is to query the source of information in an extensible way, covering a subset of all the information covered by most of the build tools. As you see, it is also strongly inspired by LSP.

@dbaeumer I'm open to collaborating with people interested in supporting BSP for any build tool. I propose a way we could move this forward in #470.

[matklad]

I think there are two somewhat independent problems here:

• The problem of logical project model (which build target does this file belong too? What are the dependencies of the build target, as build system sees it?). This is hellishly complicated and necessary language and build-system specific.

• The editor UI problem: if the user goes to the definition and ends up in the library file, what they actually see in the explorer?

I think we can tackle the second part relatively easy in a language and build-system independent manner. The simple UI solution here is to add additional read-only roots to the Explorer UI. The API could look like this:

direction: ↪️ from client to server
method: workpsace/libraryRoots
params: none
response: LibraryRoot[]

export interface LibraryRoot {
	/**
	 * The associated URI for this library root.
	 */
	uri: string;

	/**
	 * The name of the library root. Defaults to the
	 * uri's basename.
	 */
	name: string;
}

For example, for Node projects, libraryRoots would include node_modules in the parent directories, for Rust, some paths from ~/.cargo/registry/src, for Python the site-packages of the current Python.

There's a curious case of Java's JAR files, but the API covers this case as well: uri can have a scheme like jar://, and then the client could, via custom file system provider, read the Jar and draw an explorer-like tree. If add a DidChangeLibraryRoots server to client notification, we solve the refresh problem as well.

[dbaeumer]

I will close the issue since it has not gained any traction and I don't see us working on it any time soon.