[WIP] Proposal for type hierarchy

[gorkem]

Add new textDocument/subTypes and textDocument/superTypes
requests. Introduces SymbolNode.

Fixes microsoft/language-server-protocol#136

[felixfbecker]

If it's the immediate descendants, it should be called children

[gorkem]

The doc is incorrect needs to be updated.

I am not sure about children either. This can actually hold hierarchies with depth larger than one. Probably needs a better name.

[felixfbecker]

Even if it's depth larger than one, wouldn't the elements of this particular array still be the immediate children?

[gorkem]

We use this same data structure for both subTypes and superTypes calls on the superTypes call they are actually the ancestors/parents. We probably need a new name to reflect that.

[svenefftinge]

Talking about children would be natural for me, as the symbol nodes will typically be presented in tree-like structures in the client. Also, it matches with hasChildrenbelow.

[MaskRay]

It may sometimes be useful to expand multiple levels in one request, This is how ccls and cquery do.

[felixfbecker]

Is there a reason this proposal is using the term "type" everywhere and not "symbol"? For example in TypeScript, those are two very different concepts

[gorkem]

I think those are very different concepts on most languages.

[felixfbecker]

So a type can have multiple parent types? If that is true, how would it be displayed in a tree UI (vs a graph)?

[gorkem]

multiple-inheritance has been around and there have been tree based representations of it implemented before. The most generic approach I have seen is to replicate the same Type hierarchy tree under all parent nodes. We can add additional information to this data structure if it is going to help with its representation.

[felixfbecker]

I think you should reclarify the goal of this proposal, I misunderstood it. The linked issue describes a symbol hierarchy (e.g. namespaces -> classes -> members), not inheritance

[gorkem]

Also take a look at microsoft/language-server-protocol#472 I think that should include the symbol hirearchy

[felixfbecker]

I don't think this is fixes microsoft/language-server-protocol#136 - that issue is about a symbol hierachy tree in a workspace, but this seems to be about getting the inheritance hierachy for definition at a given location.

[gorkem]

microsoft/language-server-protocol#136 refers to inheritance on object-oriented languages, I know because I have written it. I think you are thinking about a way to solve the problem created by the very insufficient SymbolInfomation#containerName field, which is not this one.

[felixfbecker]

Okay, I must have remembered it wrong. Sorry for the confusion. I always had a workspace symbol request in my mind.

[dbaeumer]

Some first comments:

• shouldn't the API be more level based. What do you expect such a request returns if ask on a huge Java system for the type hierarchy of Object.
• this is somehow foreseen with the hasChildren but from the proposal it is unclear to me how a client would resolve the next level of the children.

Some ideas:

• we have textDocument/subTypes & textDocument/superTypes requests
• we have symbolNode/resolve requests for the next level.

We should also see how this aligns with the document symbol hierarchy request to generate the outline of a document.

[gorkem]

The idea was to make another call to textDocument/subTypes with the information from the Location information from SymbolNode. There isn't any generic enough information that can be passed to symbolNode/resolve that is helpful and different fromtextDocument/subTypes.

Another idea is to pass SymbolInformation to textDocument/subTypes and textDocument/superTypes calls instead of current TextDocumentPositionParams.
A new param type like below. We can also amend SymbolInformation with an optional data field similar to CompletionItem that servers would retrieve back to skip the symbol search. The downside is we need to always retrieve the SymbolInformation prior to hierarchy calls.

export interface HierarchyParams {
  /**
   * The symbol for the call.
   */
	symbol: SymbolInformation
}

[dbaeumer]

We should use the same data property mechanism that we use for document links and completion items to allow the server to identify the object in the resolve call.

I am wondering whether it makes sense to have a Tree and Graph instead of a SymbolNode in the protocol. Then the request would return a Graph. We could further classify the edges in the tree / graph. This would then work for hierarchical document symbols, type hierarchies, namespace hierarchies or even reference search.

[tsmaeder]

@dbaeumer I don't quite get what you mean with your last comment. Both about the "data property mechanism" and what you would like to do with trees and graphs.

[dbaeumer]

@tsmaeder a completion item and a document link can carry generic data property that the server can fill in and that the client preserves when calling the resolve request. That way the server can uniquely identify the completion item instance when it receives a resolve request. In JDT speak that data filed could contain the Java model identifier to identify the symbol when send back to the server for resolving children.

SymbolInformation literals are already returned in arrays. We didn't add a nextpointer to it :-) So instead if defining the tree / graph structure inside SymbolInformation or a sub type of it we could define a TreeNode and GraphNode in the LSP and then return a GraphNode for the type hierarchy. We could also type the edges to make the graph useful for containment, call hierarchies, sub type / super type, ....

[svenefftinge]

When I open a type hierarchy on a certain position in the text, I would like to see the SymbolNode for that token together with its subtypes. How do I obtain the SymbolNode I am opening the hierarchy on?

[MaskRay]

There can be multiple dimensions of sub-super relations: template parameters related to a template, derived types related to a base, specializations. Is there some way to differentiate them in this proposal?

In ccls, both derived types and specializations are stored in the IndexType::derived field.

[tsmaeder]

@dbaeumer So the GraphNode thing is more about not linking it to the SymbolInformation than any functional difference? Making it more general, but structurally it would be similar?
Got it about the "userData".

[dbaeumer]

@tsmaeder yes that is the idea.

[gorkem]

FYI since this has stalled a bit because I got new responsibilities @tsmaeder will be taking over this PR, so stay tuned

[dbaeumer]

@tsmaeder in VS Code we experimented with a GraphNode and we gave up on it :-(. Mainly because it was hard to construct a graph without adding a builder or some other kind of help methods. If you have a nice idea I am still open to it. Otherwise we found that it is better to have specific return types for every request. So in the type hierarchy case something like a TypeHierarchyNode.

[simark]

A use case I would like to see supported is calling the type hierarchy request on a method (declaration, definition or call) and seeing which nodes implement that method. In Eclipse JDT and CDT, for example (probably others too), you can get something like this:

This gives the hierarchy from the point of view of the Child2 class, but also tells you that the method I called "type hierarchy" has an definition in the Parent class, but not in the Child class.

Protocol-wise, I think that calling "type hierarchy" on a method could be equivalent to doing the request on the type of the object in which the method is declared/defined or on which the method is called. But the hierarchy nodes could also include a flag (only useful if the target of the request is a method) indicating if the node implements that method. In addition, we would need to get information about what the user has called "type hierarchy" on (name, type, etc).

[kittaakos]

As of LSP 3.10.0, we can use the DocumentSymbol as is, right? It has the children property.

[kittaakos]

We need one structural change, and we could simplify this PR:

The DocumentSymbol should be

export class DocumentSymbol {
	name: string;
	detail?: string;
	kind: SymbolKind;
	deprecated?: boolean;
	range: Range | Location;
	selectionRange: Range | Location;
	children?: DocumentSymbol[];
}

instead of the current state:

export class DocumentSymbol {
	name: string;
	detail?: string;
	kind: SymbolKind;
	deprecated?: boolean;
	range: Range;
	selectionRange: Range;
	children?: DocumentSymbol[];
}

When the range and the selectionRange has the type Range then the DocumentUri is inferred. For example, when calling textDocument/documentSymbol, the DocumentUri can be inferred as DocumentSymbolParams.textDocument.uri from the param. If the range and the selectionRange are Location types, then the DocumentUri is explicitly defined (Location.uri).

If we accept this structural change in the DocumentSymbol, we can defined our two new methods as textDocument/subTypes and textDocument/superTypes with DocumentSymbol[] return type.

We can even stick to the children name for the textDocument/superTypes; the supertypes will be real children of the DocumentSymbol node, only the way of the traversal differs.

CC: @tsmaeder (since you have taken over the PR)

[svenefftinge]

+1 See related #420 (comment).
Btw. the documentation for DocumentSymbol#selectionRange says

The range that should be selected and revealed when this symbol is being picked, e.g the name of a function.
	 * Must be contained by the `range`.

So with this contract it is enough if the range can alternatively be a Location.

[svenefftinge]

Actually, I would vote for adding an optional uri: URI property to DocumentSymbol, because the property name range suggests that the datatype is a Range, too. Also, it will not break existing clients.

[svenefftinge]

microsoft/language-server-protocol#582

[kittaakos]

I am done with a reference server implementation for Java, also with the client for Eclipse Theia. (The Java implementation of the extended LSP is here).

Type Hierarchy in action:

---

Besides declaring the textDocument/subTypes and textDocument/superTypes methods, I had to add an optional URI (uri?: DocumentUri) property to the DocumentSymbol. No other changes were invovled.

---

Since I am not allowed to make any changes in this PR, how should we proceed to speed things up a bit? Is it OK if I create a separate PR with the proposed changes? (CC: @dbaeumer, @tsmaeder)

Update:
Here is the follow-up PR: #426

[dbaeumer]

Sorry for the long delay on this. From the experiences I gained with the LSP so far I would opt to not overload any existing types. We are better off introducing new once even if they look similar or at least introduce a type alias. Otherwise it will be hard to evolve them separately. In this regard I would opt for the following:

• have a type TypeHierarchyItem
• it should be possible to lazy resolve a type hierarchy.

Given the above to I would end up with something like this:

interface TypeHierarchyItem {
	name: string;
	detail?: string;
	kind: SymbolKind;
	deprecated?: boolean;
	uri: string;
	range: Range;
	selectionRange: Range;

	parents?: TypeHierarchyItem[];
	children?: TypeHierarchyItem[];

	data?: any
}

A first request would return a TypeHierarchyItem for a given cursor location. The request would also allow to specify if the item should be resolved and whether sub or super types are to be resolved. Something like this:

interface TypeHierarchyParams extends TextDocumentPositionParams {
    resolve?: number; // the levels to resolve. 0 indicates no level.
    direction?: 'parents' | 'children' | 'both';
}

We also need another request to resolve an unresolved TypeHierarchyItem (which is indicated if parents or children is undefined. If resolved and no parents or children are available then an empty array is returned). The request would look the same as resolving a compeletion item with the same resolve and direction params as above. So something like this

interface ResolveTypeHierarchyItemParams {
    item: TypeHierarchyItem;
    resolve: number; // the levels to resolve. 0 indicates no level.
    direction: 'parents' | 'children' | 'both';
}

The optional data field can be used to identify a hierarchy item in a resolve request.

Does this make sense?

@jrieken any comments / objections on this design from a VS Code API perspective.

[dbaeumer]

Closing in favour of #426