[develop] Nix: add lsp shell

README-dev.md

@@ -23,7 +23,7 @@ Currently, Mina builds/runs on Linux & macOS. MacOS may have some issues that yo
 
 The short version:
 
-1.  Start with Ubuntu 18 or run it in a [virtual machine](https://www.osboxes.org/ubuntu/)
+1.  Start with Ubuntu 18 or run it in a virtual machine
 2.  Set github repos to pull and push over ssh: `git config --global url.ssh://[email protected]/.insteadOf https://github.com/`
     - To push branches to repos in the MinaProtocol or o1-labs organisations, you must complete this step. These repositories do not accept the password authentication used by the https URLs.
 3.  Pull in our submodules: `git submodule update --init --recursive`
@@ -46,10 +46,6 @@ Refer to [/dev](/dev).
 
 ### Developer Setup (Linux)
 
-#### Install or have Ubuntu 18
-
-- [VM Images](https://www.osboxes.org/ubuntu/)
-
 #### Building
 
 Mina has a variety of opam and system dependencies.

flake.nix

@@ -1,10 +1,12 @@
 {
-  description = "Mina, a cryptocurrency with a lightweight, constant-size blockchain";
+  description =
+    "Mina, a cryptocurrency with a lightweight, constant-size blockchain";
   nixConfig = {
     allow-import-from-derivation = "true";
     extra-substituters = [ "https://storage.googleapis.com/mina-nix-cache" ];
-    extra-trusted-public-keys =
-      [ "nix-cache.minaprotocol.org:D3B1W+V7ND1Fmfii8EhbAbF1JXoe2Ct4N34OKChwk2c=" ];
+    extra-trusted-public-keys = [
+      "nix-cache.minaprotocol.org:D3B1W+V7ND1Fmfii8EhbAbF1JXoe2Ct4N34OKChwk2c="
+    ];
   };
 
   inputs.utils.url = "github:gytis-ivaskevicius/flake-utils-plus";
@@ -59,7 +61,8 @@
       };
       pipeline = with flake-buildkite-pipeline.lib; {
         steps = flakeSteps {
-          pushToBinaryCaches = [ "s3://mina-nix-cache?endpoint=https://storage.googleapis.com" ];
+          pushToBinaryCaches =
+            [ "s3://mina-nix-cache?endpoint=https://storage.googleapis.com" ];
           signWithKeys = [ "/var/secrets/nix-cache-key.sec" ];
           commonExtraStepConfig = {
             agents = [ "nix" ];
@@ -255,7 +258,19 @@
         packages.default = ocamlPackages.mina;
 
         devShell = ocamlPackages.mina-dev;
-        devShells.default = ocamlPackages.mina-dev;
+        devShells.default = self.devShell.${system};
+
+        devShells.with-lsp = ocamlPackages.mina-dev.overrideAttrs (oa: {
+          nativeBuildInputs = oa.nativeBuildInputs
+            ++ [ ocamlPackages.ocaml-lsp-server ];
+          shellHook = ''
+            # TODO: dead code doesn't allow us to have nice things
+            pushd src/app/cli
+            dune build @check
+            popd
+          '';
+        });
+
         # TODO: think about rust toolchain in the dev shell
         packages.impure-shell =
           (import ./nix/impure-shell.nix pkgs).inputDerivation;

nix/README.md

@@ -4,9 +4,9 @@ Nix is a declarative package manager for Linux, macOS and other
 UNIX-like systems. You can read more about Nix on its [official
 website](https://nixos.org).
 
-Mina can be built using Nix, in multiple ways.
+Mina can be built using Nix, in multiple ways. Follow the steps below to get started and read the [troubleshooting section](#troubleshooting) if you encounter any problem.
 
-## TL;DR installing Nix
+## 1. Install Nix
 
 If you don't already have Nix on your machine, you can install it with
 the following command:
@@ -19,31 +19,13 @@ You may also install Nix from your distribution's official repository;
 Note however that it is preferrable you get a relatively recent
 version (⩾ 2.5), and the version from the repository may be rather old.
 
-## Note about submodules
-
-Nix will **not** fetch submodules for you. You have to make sure that
-you have the entire mina source code, including submodules, before you
-run any nix commands. This should make sure you have the right
-versions checked out (in most cases):
-
-```
-git submodule sync
-git submodule update --init --recursive
-```
-
-If you don't do this, Nix may not always yell at you right away
-(especially if all the submodule directories are present in the tree
-somehow, but not correctly filled in). It will however fail with a
-strange error during the build, when it fails to find a
-dependency. Make sure you do this!
-
-## Note about Flakes
+## 2. Enable Flakes (optional but recommended)
 
 Mina is packaged using [Nix Flakes](https://nixos.wiki/wiki/Flakes),
 which are an experimental feature of Nix. However, compatibility with
 pre-flake Nix is provided. If you wish to contribute the Nix
 expressions in this repository, or want to get some convenience
-features and speed improvements, it is advisable to enable flakes. For
+features and speed improvements, **it is advisable to enable flakes**. For
 this, you'll want to make sure you're running recent Nix (⩾2.5) and
 have enabled the relevant experimental features, either in
 `/etc/nix/nix.conf` or (recommended) in `~/.config/nix/nix.conf`:
@@ -56,46 +38,32 @@ echo 'experimental-features = nix-command flakes' > "${XDG_CONFIG_HOME-~/.config
 You can check that your flake support is working by running `nix flake
 metadata github:nixos/nixpkgs` for example.
 
-If you're using flakes, you have to run the `./nix/pin.sh` script to
+## 3. Add a nix registry entry
+
+If you're using flakes (see previous section), **you have to run the `./nix/pin.sh` script** to
 get the `mina` registry entry, since that's the easiest way to enable
 submodules to be available to the build.
 
-## Note about git LFS
+## 4. Use it
 
-If you have git LFS installed and configured on your system, the build may fail with strange errors similar to this:
+### IDE with LSP support (vscode, emacs, (neo)vim, ...)
 
-```
-Downloading docs/res/all_data_structures.dot.png (415 KB)
-Error downloading object: docs/res/all_data_structures.dot.png (fed6771): Smudge error: Error downloading docs/res/all_data_structures.dot.png (fed6771190a9b063246074bbfe3b1fc0ba4240fdc41abcf026d5bc449ca4f9b8): batch request: missing protocol: ""
+Just run
 
-Errors logged to /tmp/nix-115798-1/lfs/logs/20220121T113801.442266054.log
-Use `git lfs logs last` to view the log.
-error: external filter 'git-lfs filter-process' failed
-fatal: docs/res/all_data_structures.dot.png: smudge filter lfs failed
-error: program 'git' failed with exit code 128
-(use '--show-trace' to show detailed location information)
 ```
-
-You can fix this by setting `GIT_LFS_SKIP_SMUDGE=1` env variable, e.g. by running
-
+nix develop mina#with-lsp -c $EDITOR .
 ```
-export GIT_LFS_SKIP_SMUDGE=1
-```
-Before running any `nix` commands.
 
-## "Impure" build
+if you have your `$EDITOR` variable set correctly. Otherwise, replace it with
+the editor you want to edit Mina with.
 
-You can use Nix to only fetch the "system" (native) dependencies of
-Mina, and let `opam`, `cargo` and `go` figure out the relevant
-language-specific dependencies. To do so, run `nix-shell` (or `nix
-develop mina#impure` if you have flakes).
-
-It will drop you in a shell with all the relevant libraries and
-binaries available, and show you the instructions for building Mina.
+This command will try to `dune build @check` in `src/app/cli`, in order to get
+type information necessary for the LSP to work. This might take a while, but
+will only happen once. After it's done, you will be dropped in your favourite editor.
 
-## "Pure" build
+### "Pure" build
 
-You can also use Nix to fetch all the dependencies, and then only use
+You can use Nix to fetch all the dependencies, and then only use
 `dune` as a build system to build Mina itself.
 
 This way, you can build the entirety of Mina inside the Nix sandbox
@@ -108,9 +76,18 @@ you're using flakes). This will drop you in a shell with all
 dependencies, including OCaml, Rust and Go ones available, so the only
 thing you have to do is run `dune build src/app/cli/src/mina.exe`. You
 can also just run `eval "$buildPhase"` to run the same command as
-would be run inside the nix sandbox.
+would be run inside the nix sandbox. Note that `opam` will **not** be available in that shell, since Nix takes over the job of computing and installing dependencies. If you need to modify the opam switch, use the impure build (next section).
+### "Impure" build
 
-## Building a docker image
+You can also use Nix to only fetch the "system" (native) dependencies of
+Mina, and let `opam`, `cargo` and `go` figure out the relevant
+language-specific dependencies. To do so, run `nix-shell` (or `nix
+develop mina#impure` if you have flakes).
+
+It will drop you in a shell with all the relevant libraries and
+binaries available, and show you the instructions for building Mina.
+
+### Building a docker image
 
 Since a "pure" build can happen entirely inside the Nix sandbox, we
 can use its result to produce other useful artifacts with Nix. For
@@ -122,9 +99,11 @@ can load the image using `docker load -i result`, then note the tag it
 outputs. You can then run Mina from this docker image with `docker run
 mina:<tag> mina.exe <args>`.
 
-## Contributing to Nix expressions
 
-You probably want to [enable flakes](#note-about-flakes) if you plan
+## Miscellaneous
+### Contributing to Nix expressions
+
+You probably want to [enable flakes](#2-enable-flakes-optional-but-recommended) if you plan
 to contribute to the Nix expressions here.
 
 Most Nix things (including this README ☺) are located inside the
@@ -151,3 +130,47 @@ nixpkgs with some overlays applied (see
 [./overlay.nix](./overlay.nix)). All the dependencies are then
 provided to the final Mina derivation. See [./ocaml.nix](./ocaml.nix)
 for more details.
+
+## Troubleshooting
+
+
+### Submodules
+
+Nix will **not** fetch submodules for you. You have to make sure that
+you have the entire mina source code, including submodules, before you
+run any nix commands. This should make sure you have the right
+versions checked out (in most cases):
+
+```
+git submodule sync
+git submodule update --init --recursive
+```
+
+If you don't do this, Nix may not always yell at you right away
+(especially if all the submodule directories are present in the tree
+somehow, but not correctly filled in). It will however fail with a
+strange error during the build, when it fails to find a
+dependency. Make sure you do this!
+
+### git LFS
+
+If you have git LFS installed and configured on your system, the build may fail with strange errors similar to this:
+
+```
+Downloading docs/res/all_data_structures.dot.png (415 KB)
+Error downloading object: docs/res/all_data_structures.dot.png (fed6771): Smudge error: Error downloading docs/res/all_data_structures.dot.png (fed6771190a9b063246074bbfe3b1fc0ba4240fdc41abcf026d5bc449ca4f9b8): batch request: missing protocol: ""
+
+Errors logged to /tmp/nix-115798-1/lfs/logs/20220121T113801.442266054.log
+Use `git lfs logs last` to view the log.
+error: external filter 'git-lfs filter-process' failed
+fatal: docs/res/all_data_structures.dot.png: smudge filter lfs failed
+error: program 'git' failed with exit code 128
+(use '--show-trace' to show detailed location information)
+```
+
+You can fix this by setting `GIT_LFS_SKIP_SMUDGE=1` env variable, e.g. by running
+
+```
+export GIT_LFS_SKIP_SMUDGE=1
+```
+Before running any `nix` commands.

nix/ocaml.nix

@@ -21,12 +21,23 @@ let
     "base58"
   ] (builtins.mapAttrs (_: pkgs.lib.last) (opam-nix.listRepo external-repo));
 
-  implicit-deps = (opam-nix.opamListToQuery export.installed) // external-packages;
+  difference = a: b:
+    pkgs.lib.filterAttrs (name: _: !builtins.elem name (builtins.attrNames b))
+    a;
+
+  extra-packages = {
+    ocaml-lsp-server = "1.4.1";
+    ocaml-system = "4.11.2";
+    ppx_yojson_conv_lib = "v0.15.0";
+  };
+
+  implicit-deps = (opam-nix.opamListToQuery export.installed)
+    // external-packages;
 
   pins = builtins.mapAttrs (name: pkg: { inherit name; } // pkg) export.package;
 
-  scope = opam-nix.applyOverlays opam-nix.__overlays
-    (opam-nix.defsToScope pkgs ((opam-nix.queryToDefs repos implicit-deps) // pins));
+  scope = opam-nix.applyOverlays opam-nix.__overlays (opam-nix.defsToScope pkgs
+    ((opam-nix.queryToDefs repos (extra-packages // implicit-deps)) // pins));
 
   installedPackageNames =
     map (x: (opam-nix.splitNameVer x).name) (builtins.attrNames implicit-deps);