Allow path mapping for remote or Windows/WSL/Mingw servers

[garyo]

I run Emacs on Windows, but all the rest of my development environment is in Linux using WSL on the same machine. So it's not remote as such, but each side has its own path prefixes. Right now I'm installing the language server on the Windows side just so eglot and intellisense will work; I'd love to use the Linux-side language server, but there's a filename translation problem. Windows c:/foo/bar becomes /mnt/c/foo/bar in Linux (with default mappings anyway). If in the work you're doing here, you could allow for a filename-mapping hook (i.e. between Emacs's idea of filename and the server's idea), so I could use the Linux language server, people in my situation would be very grateful! (Note that it's no problem for Windows Emacs to start a Linux process using WSL, and get its stdio and so on. It's just the filename mapping at issue.)

The same issue happens between Windows and Mingw/Msys/Cygwin by the way; you need a simple form of pathname translation.

I suggest adding a couple of pathname-translation hooks for client-to-server and server-to-client pathname mapping.
See #348.

[joaotavora]

Can you pull up a concrete example of an LSP event exchange and show the before and after version that you would be able to pull off if you had the hooks you request?

[FelipeLema]

in the meantime, sounds like running Emacs in WSL might be a very quick way around this problem. Is there any restriction for doing this?

[joaotavora]

@FelipeLema I think this issue "holds water", i.e. merits analysis, even if there is this workaround.

[garyo]

in the meantime, sounds like running Emacs in WSL might be a very quick way around this problem. Is there any restriction for doing this?

Yes, primarily that you have to install and run a Windows X11 server (as well as install all the X client libs on the WSL side) and Windows X11 servers aren't very good and have various problems that make running Emacs that way fairly annoying. (Unless you just want to use terminal Emacs only, but that's also not ideal.)

[garyo]

Can you pull up a concrete example of an LSP event exchange and show the before and after version that you would be able to pull off if you had the hooks you request?

This may take me a while -- I did set up test language servers on WSL but didn't save the code once I saw it didn't work. I'll have to set that up again. Not all that hard, just a mental context switch or three ;-)

[FelipeLema]

someone else had a related problem which they solved at microsoft/WSL#3907 (comment)

I'm pointing this out because it seems the user was very flustered about this issue and, from the looks of the solution being presented, there's no way around it here in eglot other than adding (if (eq system-type 'windows-nt) ...) anywhere there's a filepath⇔uri translation.

[FelipeLema]

there's a wslpath.exe command? does anyone know where is all this documented?

microsoft/WSL#4704 (comment)

[garyo]

Yes, wslpath is part of WSL (it's included in Microsoft's Ubuntu distro for instance). I know, you kind of "just have to know" this stuff. :-(

% wslpath --help
wslpath: unrecognized option: -
wslpath: Invalid argument
Usage:
    -a    force result to absolute path format
    -u    translate from a Windows path to a WSL path (default)
    -w    translate from a WSL path to a Windows path
    -m    translate from a WSL path to a Windows path, with '/' instead of '\'

EX: wslpath 'c:\users'

and check e.g. https://docs.microsoft.com/en-us/windows/wsl/interop 

This week I am trying to build emacs on WSL and run an X server to see if that improves my situation with eglot. No results yet but I'll let you know.

I would think that since eglot can connect to a remote LSP process (via host/port), it, or maybe the protocol itself, would need some path-mapping anyway, right? Presumably there's no guarantee that the local filesystem is mounted the same way as the remote (where the lsp server is running), even in an all-Linux environment. Or maybe this hasn't been thought through yet?

[FelipeLema]

oh, I got confused along the way

tramp support can help you if you edit your files remotely using something like /plink: as prefix. The overhead of doing ssh to your own machine (through PuTTY) won't be noticed. Tramp will do the bridging between Windows world and the Linux world.

Worth noting that using a Virtualbox with a guest Linux machine + mounting vbox shares would be much easier than compiling emacs with X in WSL. I'm using this setup at work for the same reasons, although I'm editing remote files through ssh.

You can try this setup while we wait for #463 as it looks like the best candidate solution so far to this issue.

[dandavison]

I have the following setup, which I think ends up imposing the same requirements as @garyo described in the intro to this ticket: basically eglot needs a hook to make an arbitrary transformation to a file path.

The setup:

1. Emacs runs on the host OS
2. The application being developed runs in a docker container
3. 3rd-party source code (a python virtualenv) is stored in a docker volume and is not present on the host filesystem
4. 1st-party source code is present on the host filesystem, and mounted as a volume inside the docker container

Note that

• Because of (3), the LSP server process must run in the docker container.
• However, I am mostly not using Tramp: when editing the 1st-party code, an Emacs running in the host OS is accessing files in the host filesystem.
• docker-tramp exists which allows a tramp-style file path like /docker:a4b65b31ef15:/home/docker/virtualenv/some_package/file.py to be used in the host Emacs process to read/write to the file in the docker volume.

So the typical situation is that the host Emacs process is visiting a file on the host OS, not using Tramp, but we want eglot to be able to jump to the definition of something in 3rd-party code. This requires eglot to translate the file path received from the server into a docker-tramp style file path. But unlike #463 it cannot use file-remote-p to do this, since the file that Emacs is visiting is not remote.

I needed something quick to be able to work, so for what it's worth here's what I'm currently using. (This is just a quick hack but I'm happy to help out with making the proper changes to eglot to do the various things being requested regarding tramp and remote paths.)

Change to eglot

commit 48566594aeef5718aaa17ff9625672b0ad5af8c6
Author: Dan Davison <[email protected]>
Date:   Fri Apr 24 20:07:57 2020 -0400

    Add user variable eglot-transform-path-function

diff --git a/eglot.el b/eglot.el
index 8dae0b7..51d0524 100644
--- a/eglot.el
+++ b/eglot.el
@@ -203,6 +203,16 @@ let the buffer grow forever."
   :type '(choice (const :tag "Don't show confirmation prompt" nil)
                  (symbol :tag "Show confirmation prompt" 'confirm)))
 
+(defcustom eglot-transform-path-function #'identity
+  "A function to transform a file path received from the server into the
+appropriate path for the client.
+
+This can be used when the server and the client do not share the
+same filesystem, for example to transform the path received from
+the server into a tramp-style path which the client can use to
+read and write the remote file."
+  :type 'function)
+
 
 ;;; Constants
 ;;;
@@ -1886,7 +1896,7 @@ Calls REPORT-FN maybe if server publishes diagnostics in time."
   "Like `xref-make-match' but with LSP's NAME, URI and RANGE.
 Try to visit the target file for a richer summary line."
   (pcase-let*
-      ((file (eglot--uri-to-path uri))
+      ((file (funcall eglot-transform-path-function (eglot--uri-to-path uri)))
        (visiting (or (find-buffer-visiting file)
                      (gethash uri eglot--temp-location-buffers)))
        (collect (lambda ()

My personal config:

(defun dan/eglot-pyls-transform-path-to-docker-tramp (path)
  (cl-destructuring-bind (container-id)
      (cl-loop for (id name) in (docker-tramp--running-containers)
               if (s-contains-p "_pyls_" name)
               collect id)
    (format "/docker:%s:%s" container-id path)))

(setq eglot-transform-path-function #'dan/eglot-pyls-transform-path-to-docker-tramp)

[dandavison]

What I suggest is:

1. We wait for the tramp work in Tramp support, fixes #84 #463 to settle down (maybe it is stable enough already, I'm not sure).

2. We then add an additional customization entrypoint allowing users to supply an arbitrary filepath-transformation function. For example, something like the funcall-based solution I gave in my previous message in this issue.

If anyone has any doubts about the appropriateness/necessity of this, or about that style of solution, then that would be helpful to know.

I hope my description above didn't seem too complex/niche -- it's a common situation with docker-based workflows. In fact, it was the difficulty of solving that without LSP that brought me to eglot.

[garyo]

I think the mapper has to work both ways, btw -- Emacs needs to be able to tell the server where to find the file, and to interpret paths/URIs coming from the server. At least that's my understanding.

[dandavison]

I think the mapper has to work both ways

Yes, good point, agreed. (My code example was just the minimum necessary that got one thing working with my particular setup -- jump-to-definition of something that is defined in a file that is accessible to the server process but not on the same filesystem as Emacs.)

[joaotavora]

I think the mapper has to work both ways, bt

Yes exactly. In SLIMM or SLY (two Common Lisp IDE's with a lot in common with LSP), it's done via a variable. Have a look here:

https://github.com/joaotavora/sly/blob/master/contrib/sly-tramp.el

[valignatev]

I'm having exactly the same setup as dandavison - i.e. my own code is on the host machine, and dependencies are in the container. I wonder if anything changed throughout the years that made a setup like this conveniently possible.

Sorry if it's better to not continue discussion in already open issues and it's better to move them to emacs-devel

[FelipeLema]

after all these years I can only suggest using Tramp as a last resource: works great if you leave it untouched, but customizing and debugging is a lot of work (and probably not worth the effort).

Try mounting paths from the container in the host machine so Emacs can see them.

[joaotavora]

Also, after 4 years, I have to admit I don't understand what the problem is about anymore. I have to re-read the comments.

I'm tempted to say that TRAMP support (which didn't exist back in 2019) is a good fix for this, but I have to re-learn the problem.