Expose build function to make custom derivations

[terlar]

It would be nice if the build function was exposed so one could make custom derivations:
https://github.com/emacs-twist/twist.nix/blob/master/pkgs/emacs/build/default.nix

The use case is that I want to build my configuration and not install it as a package but still would like to byte compile it. Perhaps the correct thing is to make packages out of this not sure, but at least that is what I am doing currently.

I have achieved this by a bit of a hack using the trivialBuild instead:

final: prev: {
  emacsEnv =
    (final.emacsTwist {
      emacsPackage = final.emacsPgtkNativeComp.overrideAttrs (_: {version = "29.0.50";});

      initFiles = [(final.tangleOrgBabelFile "init.el" ./init.org {})];

      lockDir = ./lock;
      inventories = import ./nix/inventories.nix {
        inherit self;
        emacsSrc = final.emacsPgtkNativeComp.src.outPath;
      };

      inputOverrides = import ./nix/inputOverrides.nix;
    })
      .overrideScope' (tfinal: tprev: {
        elispPackages = tprev.elispPackages.overrideScope' (
          prev.callPackage ./nix/packageOverrides.nix {inherit (tprev) emacs;}
        );
      });

  emacsConfig = let
    emacs = let
      self =
        final.emacsEnv
        // {
          inherit (final.emacsEnv.emacs) meta;
          overrideAttrs = _: self;
        };
    in
      self;

    attrs = nixpkgs.lib.optionalAttrs (self ? lastModifiedDate) {
      version = nixpkgs.lib.substring 0 8 self.lastModifiedDate;
    };
  in
    (prev.emacsPackagesFor emacs).callPackage ./. attrs;
}

Then later it is used in custom packages:

trivialBuild {
  pname = "config-init";
  inherit version;

  src = lib.sourceByRegex ./. ["init.org" "lisp" "lisp/.*.el$"];

  buildPhase = ''
    emacs --batch --quick \
      --load org \
      *.org \
      --funcall org-babel-tangle

    mkdir -p .xdg-config
    ln -s $PWD .xdg-config/emacs
    export XDG_CONFIG_HOME="$PWD/.xdg-config"

    emacs -L . --batch --eval '(setq byte-compile-error-on-warn t)' -f batch-byte-compile *.el
  '';
}

[akirak]

Not sure this is what you want, but you can build a single elisp package:

nix build .#emacs-config.elispPackages.dash

Exposing packageEnv in pkgs/emacs/wrapper.nix may be a good idea. It will allow the user to build his/her whole set of elisp packages. There was a user who suggested installation of packages to a Nix profile (emacs-twist/elisp-helpers#31). I'm not thinking of supporting non-standard elisp package managers (e.g. straight, elpaca, etc.), but it's possible to install elisp packages to a Nix profile, once the change is made.

[terlar]

I want to compile my init.el via the build function provided by twist.nix.

Also if I want to make these local sources packages how do I go about that? I saw you added something extra to inputOverrides:
https://git.sr.ht/~akirak/nix-config/tree/master/item/emacs/overlay.nix#L68

However, if I defined a local source there nothing happened, and if I added that package to extraPackages it was not found.

I want to:

• Byte compile/native compile my init.el (with all existing packages available via wrapper)
• Byte compile/native compile my local custom packages not distributed yet under my lisp folder.

The latter one I guess I could achieve by using custom recipes, but I would like to grab the source from the current flake.

[akirak]

Now I've got your points. Thank you for your patience.

• Byte compile/native compile my init.el (with all existing packages available via wrapper)

Byte-compiling (or native-compiling) initFiles passed to emacsTwist function is not specifically supported yet. I thought the performance gain would be negligible, and it would entail recompiling the whole init file (which often spans 10k lines) every time you tweak the file.

I didn't work on it because I wasn't sure if it would be a good idea, but it is possible to add the feature from now.

• Byte compile/native compile my local custom packages not distributed yet under my lisp folder.

You have to

• Add MELPA recipes for your packages
• Override their inputs to fetch from inputs.self
• Specifying the packages in your init file with :ensure set to t, or add them to extraPackages

Adding a recipe for a package means, if the package is specified in your init file, twist fetches source from somewhere specified in the recipe and build it. But you can override the source to anywhere by setting src (which can be a Nix path to the root of a repository) in the input overrides.

In your case, you can simply set src to inputs.self.outPath (where inputs is the inputs of your flake), but I use nix-filter to pass only relevant files. That should be part of your inputOverrides.

Now I think the configuration API of emacsTwist function is not high-level enough for configuration needs. The example (which is actually a wrapper of the function) may be a better fit. I am thinking of using the wrapper in #61.

[terlar]

@akirak So I finally got around to try this (the nixpkgs trivialBuild solution recently broke). It seems everything worked smoothly.

Relevant parts:

{
  emacs-config = prev.callPackage inputs.self {
    buildElispPackage = (inputs.twist.lib.buildElispPackage final).override {
      emacs = emacsPackage;
    };

    elispInputs = prev.lib.pipe final.emacs-env.elispPackages [
      builtins.attrValues
      (builtins.filter prev.lib.isDerivation)
    ];
  };
}

buildElispPackage {
  ename = "config-init";

  src = lib.sourceByRegex ./. [ "init.org" ];
  files = [ "init.org" ];
  lispFiles = [
    "early-init.el"
    "init.el"
  ];

  inherit elispInputs;
  nativeCompileAhead = true;
  wantExtraOutputs = false;
  errorOnWarn = true;
  doTangle = false;

  preBuild = ''
    export HOME="$NIX_BUILD_TOP/.home"
    mkdir -p "$HOME/.config/emacs"

    emacs --batch --quick \
      --load org \
      *.org \
      --funcall org-babel-tangle
    rm *.org

    ln -s ${tree-sitter}/lib "$HOME/.config/emacs/tree-sitter"
  '';

  meta = { };
}

Regarding the interface, here are some thoughts:

• Is it really nice to provide the emacs via override, perhaps expose that via the normal attrs? Had a brief moment of total collapse until I realized the Emacs versions was "ancient". I think this is mainly an issue with the exposed lib function as it hides the callPackage call.
• Seems errorOnWarn is hardcoded, that should probably be in the attrs
• I had to provide doTangle perhaps should default to false?
• Same thing for wantExtraOutputs.
• Should meta perhaps have some defaults (empty attrset)?
• I saw a note about autoloads being deprecated, perhaps it should be possible to opt-out from that as well, if that is the case.

All in all, really pleased with this, so thank you very much :)

[akirak]

@terlar Thanks. I'll study your config to explore how I can improve the API.

Regarding the defaults, I'm thinking of adding a convenient option to toggle errorOnWarn, doTangle, wantExtraOutputs, etc. It could be named, say, presetForLinting, but any idea is welcome.