Unable to find or run nvidia-smi despite verified presence/executability of binary

[harryaskham]

Bug description

Attempting to run nvidia-smi results in zsh: command not found: nvidia-smi. I have verified through various ways that the binary exists, is in the PATH, is chmod'd to be executable by user, but cannot be found in zsh, bash or via nix-shell.

To Reproduce

Steps to reproduce the behavior:

• Open new shell
• run nvidia-smi

Expected behavior
nvidia-smi is found on the PATH and runs

Logs

Please excuse my formatting. I will also attach a screenshot.

    ~  nvidia-smi                                                                                   127 ✘  11:25:14  
zsh: command not found: nvidia-smi

    ~  where nvidia-smi                                                                             127 ✘  11:25:18  
/usr/lib/wsl/lib/nvidia-smi

    ~  ls -lsa `where nvidia-smi`                                                                       ✔  11:25:23  
632 -r-xr-xr-x 1 root root 644048 Apr 11 12:57 /usr/lib/wsl/lib/nvidia-smi

    ~  /usr/lib/wsl/lib/nvidia-smi                                                                      ✔  11:25:32  
zsh: no such file or directory: /usr/lib/wsl/lib/nvidia-smi

    ~  sudo !!                                                                                      127 ✘  11:25:44  

    ~  sudo /usr/lib/wsl/lib/nvidia-smi                                                             127 ✘  11:25:47  
sudo: unable to execute /usr/lib/wsl/lib/nvidia-smi: No such file or directory

[SuperSandro2000]

Linux returns no such file or directory if the shebang or link loader of a file/binary is not available. I think /usr/lib/wsl/lib/nvidia-smi needs patching to work on NixOS or must be started in a FHS.

[nzbr]

I'm able to at least run it. It's a foreign binary (meaning non-nixos), so it probably doesn't find ld-linux.so. You can try setting up nix-ld to run it without patching

ZSH C:\Users\nzbr> /usr/lib/wsl/lib/nvidia-smi
NVIDIA-SMI couldn't find libnvidia-ml.so library in your system. Please make sure that the NVIDIA Display Driver is properly installed and present in your system.
Please also try adding directory that contains libnvidia-ml.so to your system PATH.

[nzbr]

I can now confirm, that it should indeed work with nix-ld, when NIX_LD_LIBRARY_PATH=/usr/lib/wsl/lib is set

[harryaskham]

Thank you for investigating; with nix-ld enabled, and export NIX_LD=/lib64/ld-linux-x86-64.so.2 and NIX_LD_LIBRARY_PATH=/usr/lib/wsl/lib, nvidia-smi is giving me:

cannot execute nvidia-smi: mmap segment of /lib64/ld-linux-x86-64.so.2 failed:
[1]    3134 segmentation fault  nvidia-smi

However if I set up a barebones shell.nix with e.g.

with import <nixpkgs> {};
mkShell {
  NIX_LD_LIBRARY_PATH = lib.makeLibraryPath [
    stdenv.cc.cc
  ];
  NIX_LD = lib.fileContents "${stdenv.cc}/nix-support/dynamic-linker";
}

Then nvidia-smi will work inside that shell, since ld-linux-x86-64.so.2 is now being sourced from the Nix store and LD_LIBRARY_PATH points to the store as well. Could you please confirm whether this was the approach you took to get it working with nix-ld? (maybe via nix-alien/autobahn)

Thanks

[nzbr]

That's what I have in regards to nix-ld in my config

{
    programs.nix-ld.enable = true;
    environment.variables = {
      NIX_LD_LIBRARY_PATH = lib.makeLibraryPath [
        pkgs.stdenv.cc.cc
      ];
      NIX_LD = "${pkgs.glibc}/lib/ld-linux-x86-64.so.2";
    };
}

I can then run nvidia-smi like this

NIX_LD_LIBRARY_PATH=/usr/lib/wsl/lib/ /usr/lib/wsl/lib/nvidia-smi

[harryaskham]

That's helpful - thanks for sharing!