nixos/no-x-libs: add more replacements #204287

Artturin · 2022-12-03T13:34:44Z

all-packages: change -nox overrides to callPackage's so there is no infinite recursion when replacing the package with a overlay

Description of changes

Things done

RaitoBezarius · 2022-12-03T14:48:35Z

@Artturin Is there an easy way to measure the delta in the closure size?

Artturin · 2022-12-03T16:49:55Z

@Artturin Is there an easy way to measure the delta in the closure size?

this doesn't change the closure size much (atleast for a minimal-ish system assuming i did the comparison correctly) but instead reduces the dependency tree size ( reducing compilation times if compiling from ex staging)

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/refs/pull/204287/head";
    # commit before the above pr
    nixpkgs-before.url = "github:NixOS/nixpkgs/7bfceb02093f2c7ea0ea7e353c99b26e9a4022fc";
  };

  outputs = inputs:
    let
      pkgs = import inputs.nixpkgs { system = "x86_64-linux"; };
      pkgs-before = import inputs.nixpkgs-before { system = "x86_64-linux"; };

      node = pkgs.nixos ({ modulesPath, ... }: {
        fileSystems."/".device = "ignore-root-device";
        boot.loader.grub.enable = false;
        imports = [
          (modulesPath + "/profiles/minimal.nix")
        ];
        networking.networkmanager = {
          enable = true;
          plugins = pkgs.lib.mkForce [];
        };
      });
      node-before = pkgs-before.nixos ({ modulesPath, ... }: {
        fileSystems."/".device = "ignore-root-device";
        boot.loader.grub.enable = false;
        imports = [
          (modulesPath + "/profiles/minimal.nix")
        ];
        networking.networkmanager = {
          enable = true;
          plugins = pkgs.lib.mkForce [];
        };
        nixpkgs.overlays = [
          (_: super: {
           # had to add this because the compilation was broken before
            stoken = super.stoken.override { withGTK3 = false; };
          })
        ];
      });

      closureInfo = pkgs.closureInfo { rootPaths = [ node.config.system.build.toplevel ]; };
      closureInfoBefore = pkgs.closureInfo { rootPaths = [ node-before.config.system.build.toplevel ]; };
    in
    {
      packages.x86_64-linux.default = pkgs.runCommand "compare"
        {
          inherit closureInfo closureInfoBefore;
          nativeBuildInputs = [ pkgs.nodePackages.insect ];
        } ''
        sizeBytes=$(cat $closureInfo/total-nar-size)
        sizeBeforeBytes=$(cat $closureInfoBefore/total-nar-size)
        insect "$((sizeBeforeBytes - sizeBytes)) bytes -> megabytes"
        touch $out
      '';
      packages.x86_64-linux.has-libX = closureInfoBefore;
      packages.x86_64-linux.no-libX = closureInfo;
    };
}

nix path-info -rsSh ".#packages.x86_64-linux.has-libX" --derivation | wc -l
2167
$ nix path-info -rsSh ".#packages.x86_64-linux.no-libX" --derivation | wc -l
1866

SuperSandro2000 · 2022-12-04T09:19:34Z

nixos/modules/config/no-x-libs.nix

+      # translateManpages -> perlPackages.po4a -> texlive-combined-basic -> texlive-core-big -> libX11
+      util-linux = super.util-linux.override { translateManpages = false; };
+      # dep of graphviz, libXpm is optional for Xpm support
+      gd = super.gd.override { libXpm = null; };


Can we please add an option instead of overwriting packages to null? This would for example break if we access libXpm.version.

Suggested change

gd = super.gd.override { libXpm = null; };

gd = super.gd.override { withXorg = false; };

pkgs/top-level/all-packages.nix

SuperSandro2000 · 2022-12-04T09:20:55Z

pkgs/top-level/all-packages.nix

    libX11 = null;
    libGL = null;


Let me create a PR that adds a proper option for this.

nixos/modules/config/no-x-libs.nix

SuperSandro2000 · 2022-12-04T21:59:56Z

nixos/modules/config/no-x-libs.nix

+      # translateManpages -> perlPackages.po4a -> texlive-combined-basic -> texlive-core-big -> libX11
+      util-linux = super.util-linux.override { translateManpages = false; };
+      # dep of graphviz, libXpm is optional for Xpm support
+      gd = super.gd.override { libXpm = null; };


Suggested change

gd = super.gd.override { libXpm = null; };

gd = super.gd.override { withXorg = false; };

pkgs/top-level/all-packages.nix

SuperSandro2000 · 2022-12-05T01:43:17Z

FYI you motivated to try this again especially for my simd branch and I created the following PRs:

roberth · 2022-12-07T11:38:01Z

ghostscript_headless: init #204977

I'm not doing this through the NixOS no-x-libs overlay, because this should improve many packages that have no business having graphical libraries in their build closure, saving many rebuilds, regardless of whether the user wants to optimize for a non-graphical system configuration.

Artturin · 2022-12-10T19:54:29Z

wait until https://nixpk.gs/pr-tracker.html?pr=204426 is in master

SuperSandro2000 · 2022-12-22T00:42:02Z

Why not merge this into staging?

nixos/modules/config/no-x-libs.nix

SuperSandro2000 · 2023-01-18T17:17:16Z

Needed to create a new PR because force pushing that one almost deadlocked my computer. #211428

all-packages: change -nox overrides to callPackage's so there is no infinite recursion when replacing the package with a overlay

Izorkin · 2023-01-20T19:56:07Z

This PR causes this error:

error: infinite recursion encountered

       at /home/user/works/src-nix/nixpkgs/pkgs/top-level/all-packages.nix:20695:3:

        20694|
        20695|   libdevil-nox = libdevil.override {
             |   ^
        20696|     inherit (darwin.apple_sdk.frameworks) OpenGL;
(use '--show-trace' to show detailed location information)

Artturin · 2023-01-20T19:57:48Z

nixos/modules/config/no-x-libs.nix

      imagemagick = super.imagemagick.override { libX11Support = false; libXtSupport = false; };
      imagemagickBig = super.imagemagickBig.override { libX11Support = false; libXtSupport = false; };
+      libdevil = super.libdevil-nox;


libdevil -> libdevil-nox -> libdevil -> libdevil-nox ...

changing this back to

libdevil-nox = callPackage ../development/libraries/libdevil { inherit (darwin.apple_sdk.frameworks) OpenGL; withXorg = false; };

will work

Thanks, working!

Ahh, sorry, messed this up when rebasing.

Thanks @SuperSandro2000

…xOS#204287

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Dec 3, 2022

ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Dec 3, 2022

RaitoBezarius approved these changes Dec 3, 2022

View reviewed changes

SuperSandro2000 reviewed Dec 4, 2022

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Dec 4, 2022

View reviewed changes

mweinelt added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label Dec 4, 2022

SuperSandro2000 reviewed Dec 4, 2022

View reviewed changes

roberth added the 6.topic: closure size The final size of a derivation, including its dependencies label Dec 7, 2022

SuperSandro2000 force-pushed the addmorenox branch 2 times, most recently from eccd9f5 to b7afcc1 Compare December 9, 2022 10:33

Artturin force-pushed the addmorenox branch 2 times, most recently from 2be05f1 to 2cff6fd Compare December 10, 2022 19:53

Artturin marked this pull request as draft December 10, 2022 19:54

SuperSandro2000 force-pushed the addmorenox branch from 2cff6fd to 2c01d5f Compare December 19, 2022 21:34

nagy reviewed Dec 25, 2022

View reviewed changes

nixos/modules/config/no-x-libs.nix Show resolved Hide resolved

SuperSandro2000 marked this pull request as ready for review January 18, 2023 16:13

SuperSandro2000 mentioned this pull request Jan 18, 2023

nixos/no-x-libs: add more replacements #211428

Closed

13 tasks

nixos/no-x-libs: add more replacements

9a34bb2

all-packages: change -nox overrides to callPackage's so there is no infinite recursion when replacing the package with a overlay

SuperSandro2000 force-pushed the addmorenox branch from 2c01d5f to 9a34bb2 Compare January 19, 2023 23:51

SuperSandro2000 approved these changes Jan 20, 2023

View reviewed changes

SuperSandro2000 merged commit fe34a63 into NixOS:master Jan 20, 2023

Artturin commented Jan 20, 2023

View reviewed changes

SuperSandro2000 added a commit to SuperSandro2000/nixpkgs that referenced this pull request Jan 20, 2023

nixos/no-x-libs: fix infinite recursion accidentally introduced in Ni…

604d6e4

…xOS#204287

Artturin mentioned this pull request Feb 15, 2023

nixos/no-x-libs: fix infinite recursion with ffmpeg #216385

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/no-x-libs: add more replacements #204287

nixos/no-x-libs: add more replacements #204287

Artturin commented Dec 3, 2022

RaitoBezarius commented Dec 3, 2022

Artturin commented Dec 3, 2022 •

edited

Loading

SuperSandro2000 Dec 4, 2022

SuperSandro2000 Dec 4, 2022

SuperSandro2000 Dec 4, 2022

SuperSandro2000 Dec 4, 2022

SuperSandro2000 Dec 4, 2022

SuperSandro2000 Dec 4, 2022

SuperSandro2000 commented Dec 5, 2022

roberth commented Dec 7, 2022

Artturin commented Dec 10, 2022

SuperSandro2000 commented Dec 22, 2022

SuperSandro2000 commented Jan 18, 2023

Izorkin commented Jan 20, 2023 •

edited

Loading

Artturin Jan 20, 2023

Artturin Jan 20, 2023 •

edited

Loading

Izorkin Jan 20, 2023

SuperSandro2000 Jan 20, 2023

SuperSandro2000 Jan 20, 2023

Artturin Jan 21, 2023

	gd = super.gd.override { libXpm = null; };
	gd = super.gd.override { withXorg = false; };

nixos/no-x-libs: add more replacements #204287

nixos/no-x-libs: add more replacements #204287

Conversation

Artturin commented Dec 3, 2022

Description of changes

Things done

RaitoBezarius commented Dec 3, 2022

Artturin commented Dec 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SuperSandro2000 commented Dec 5, 2022

roberth commented Dec 7, 2022

Artturin commented Dec 10, 2022

SuperSandro2000 commented Dec 22, 2022

SuperSandro2000 commented Jan 18, 2023

Izorkin commented Jan 20, 2023 • edited Loading

Choose a reason for hiding this comment

Artturin Jan 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Artturin commented Dec 3, 2022 •

edited

Loading

Izorkin commented Jan 20, 2023 •

edited

Loading

Artturin Jan 20, 2023 •

edited

Loading