Completion with non-prefix candidates

[astoff]

It would be nice to support non-prefix matches for completion candidates. Currently, it seems that all non-prefix matches returned by the server are discarded, when using either company or completion-at-point.

In latex, for instance, completions after \doc should include \documentclass and \begin{document}. Even if the response from the server (attached below) includes both, with the property filterText appropriately set, eglot suggests only \documentclass.

There is at least one further use case for this feature in latex: completing \cite{label could suggest not only bibtex entries whose identifier starts with label, but also any entries fuzzy-matching 'label`. This would allow using bibtex entries with non-mnemonic labels, like those one can download from indexing services.

client-request (id:2) Mon Dec 10 10:07:57 2018:
(:jsonrpc "2.0" :id 2 :method "textDocument/completion" :params
          (:textDocument
           (:uri "file:///home/augusto/y.tex")
           :position
           (:line 0 :character 4)))

server-reply (id:2) Mon Dec 10 10:07:57 2018:
(:jsonrpc "2.0" :id 2 :result
          [(:insertTextFormat 1 :textEdit
                              (:newText "documentclass" :range
                                        (:start
                                         (:character 1 :line 0)
                                         :end
                                         (:character 4 :line 0)))
                              :label "documentclass" :detail "[options]{class}")
           (:insertTextFormat 1 :textEdit
                              (:newText "begin{document}" :range
                                        (:start
                                         (:character 1 :line 0)
                                         :end
                                         (:character 4 :line 0)))
                              :filterText "document" :label "begin{document}")])

[joaotavora]

Ok, this is very interesting. I have since something similar for Sly recently and indeed I do miss it in Eglot.

I recently added special support for this kind of stuff in company-capf that hopefully should help us there. In bare completion-at-point it's probably a bit harder (Sly has a lot of adhoc code for that)

[joaotavora]

Can you confirm it's something like https://github.com/joaotavora/sly/blob/master/doc/animations/company-flex-completion.gif you are after?

[mkcms]

It would also be useful for example with clangd, which, for some candidates, prepends a special prefix to completion label - the character •. It is impossible to insert such candidates when there's something before point. They can only be inserted when completing without a prefix.

(:additionalTextEdits
		      [(:newText "#include \"Market.h\"\n" :range
				 (:end
				  (:character 0 :line 17)
				  :start
				  (:character 0 :line 17)))]
		      :detail "\n\"Market.h\"" :filterText "Market" :insertText "Market" :insertTextFormat 2 :kind 7 :label "•Market" :sortText "3ff8a156Market" :textEdit
		      (:newText "Market" :range
				(:end
				 (:character 4 :line 35)
				 :start
				 (:character 4 :line 35))))

[astoff]

@joaotavora Yes, something like the linked gif would do the job.

[joaotavora]

@astoff it would be very useful to have an easy to run server that supports this kind of non-prefix completion. What latex server are you using?

@mkcms is this only on clangd-8?

[mkcms]

@mkcms is this only on clangd-8?

I don't know about that (I am using clangd-8). The code says it's "A visual indicator to prepend to the completion label to indicate whether completion result would trigger an #include insertion or not".

https://github.com/llvm-mirror/clang-tools-extra/blob/48f63c66dd1e56d142ce05b01bc0f1ae7dde1065/clangd/CodeComplete.h#L71

Maybe you need to build clangd index to enable it, since I'm doing that.

[astoff]

@astoff it would be very useful to have an easy to run server that supports this kind of non-prefix completion. What latex server are you using?

I'm using this one. It is certainly easy to run, and I can implement the citation fuzzy matching soon for another interesting test case. On the downside, it's mostly a sketch at this point, and I have tested it only with Eglot. It would be better to make sure first that my server plays well with clients that already do non-prefix matching; unfortunately it doesn't seem so easy to just run your own server on the "reference client", if you would call VS Code so.

[joaotavora]

I'm using this one. It is certainly easy to run, and I can implement the citation fuzzy matching soon for another interesting test case.

Oh it looks very good. I wish I had this when I was recently writing my master thesis

On the downside, it's mostly a sketch at this point, and I have tested it only with Eglot.

❤️

Unfortunately, on debian buster/sid, luarocks complains that it only finds lua 5.1. I installed lua 5.3 and and even added it to the alternatives, but I still can't install your server :-(. Any help?

[astoff]

It's funny how the dependencies work on Debian, but the following works for me (on Ubuntu 18.04): apt install lua5.3 liblua5.3-dev luarocks. It's probably good to make sure first that lua5.1 or lua5.2 are not installed to avoid fiddling with alternatives.

[joaotavora]

It's funny how the dependencies work on Debian, but the following works for me (on Ubuntu 18.04): apt install lua5.3 liblua5.3-dev luarocks. It's probably good to make sure first that lua5.1 or lua5.2 are not installed to avoid fiddling with alternatives.

Looks line "funny" == "broken" in this case. Anyway, I did what you suggested and it worked. So got it working. I'll let you know once I come up with something.

[joaotavora]

@astoff for this to work correctly we need support from the LSP specification.

Here's a preamble, that you might want to skip. Basically, completions in LSP are very client agnostic: they work like "Here's a document, here's point, optionally here's what's been pressed recently, now give me the completions", whereas Emacs normally works like "here's a substring and some minimal context, give me strings that complete the substring". It's not easy to judge what is better, and we sure aren't going to change the basic workings of the LSP approach, but it makes it difficult for Emacs to guess exactly what string the LSP server is completing. But this doesn't necessarily matter here.

For this to work, the server has to provide the regions of each completion that it thinks are the parts of that that are already set down. So the user requests completion after the word afoo in the document and the server decides to provides completions farfromsober and galaxyfooraway. If it does so it should also say somewhere the numbers (1 3 5 8) for the first completion and (1 6 7 8) for the second completion: these are the 0-based indexes of the characters of afoo in the completion. Or maybe it can say (1 3 5 8) for the first and (1 (6 8)) for the second to save bandwidth. Note that all of this is indeed consistent with the user wanting to complete afoo, but it's up to the server to decide which a in galaxyfooraway it means.

Well, long story short, we need a new field for the Completion data type. We can experiment with a new experimental $matchIndexes field. Alternatively, we can use the existing data field, whose datatype is any. Maybe to JSONize this it's easier [1 3 5 8] for the first example and [1 {:start 6 :end 8}] for the second example.

Of course, I'll create an issue in the LSP github page.

[nemethf]

I've just reread the original issue description and I think there's a different (easier) solution to the problem. I think the culprit is that eglot does not follow closely the lsp standard when it comes to filterText properties.

However, I believe a workaround can be implemented in the server. I haven't tested it, but if the server sends "document" as label instead of "begin{document}", then eglot inserts the newText property ("begin{document}") correctly.

Obviously, the correct solution is to support filterText in eglot. I made a half-baked attempt to do that in #235, but it might not solve this issue.

Having said that, flex matching would be awesome.

[joaotavora]

eglot does not follow closely the lsp standard when it comes to filterText properties.

Although I won't have time to review it anytime soon, please explain exactly how, or open a new issue.

In case #235 is already that issue, disregard this.

[astoff]

After a long time — thanks for looking into this and also bringing up the issue with the LSP people. I guess it will just take a while for the specs to mature and resolve this question completely.

In any case, would you like to set up eglot to work out of the box with digestif? I think it does the basic stuff well enough by now. It should be enabled in tex-mode, latex-mode and context-mode, and the executable name is digestif, no extra args required.

[astoff]

In version 1.5, flex matching works when filterText is appropriately set, so I'll close this.