libpq: init at 17.2

[wolfgangwalther]

This creates a separate libpq package as discussed in #61580 (comment).

This brings down the number of rebuilds after updating postgresql to about 1.2k darwin and 2.5k linux. Before, they were ~ around 5k, IIRC.

Unfortunately, we are still not anywhere low enough to be able to merge postgresql updates directly into master. I think the nr. 1 reason for that is postgresqlTestHook - and the fact that many packages use a very simple postgresql server in their build dependencies for the check phase.

The only way to get the number of rebuilds lower would be to package postgresqlTestHook as a separate postgresql derivation. The idea would be, that security related issues are irrelevant for the check phase of other packages - and thus updating this separately could easily go through staging with a certain delay. This derivation could also be one that is built with the minimal feature set, making it slimmer overall.

I still think that introducing libpq is a good thing on it's own, thus this PR.

Closes #61580 and #191920

Things done

• Built on platform(s)
  • x86_64-linux (the directly changed packages)
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[Ma27]

I promise, I haven't forgotten about that, I'll do my best to schedule a review soon! :))

[Ma27]

FYI pushed a fix for pdo_pgsql/pgsql exts in PHP.

[wolfgangwalther]

FYI pushed a fix for pdo_pgsql/pgsql exts in PHP.

Seems like that needs some nixfmt, I don't get why, though.

[Ma27]

Fixed.

[Ma27]

The change itself looks good to me.

After re-reading #294504 (comment) I'm not sure if I consider the status quo that bad: packages like psycopg or pdo_pgsql only depend on the lib output at runtime which is effectively libpq already. The static build seems to produce static libraries successfully as well nowadays. Regarding dependency concerns, one may still work with outputChecks.

OTOH with another postgresql package that's then used for the test hook we could allow shipping all server updates to master directly. But then again, we could also ship security updates to staging-next directly to speed things up a bit.

I'm not sure.
I'm slightly leaning towards this solution, but figured it'd make sense to share my thoughts from review before merging (hoping that we didn't discuss this before and I just didn't find the thread again), not after.

[szlend]

Apologies, misclicked 😅

[wolfgangwalther]

After re-reading #294504 (comment) I'm not sure if I consider the status quo that bad: packages like psycopg or pdo_pgsql only depend on the lib output at runtime which is effectively libpq already. The static build seems to produce static libraries successfully as well nowadays. Regarding dependency concerns, one may still work with outputChecks.

Dependencies come into play whenever we need to (re-)build. For us the most relevant case is the one inside nixpkgs - where we discuss how to speed up merging to master etc.

However, there's more: Yes, postgresql builds in pkgsStatic now. But it still carries more dependencies. All of which might need to be rebuild when you're trying to actually cross-compile libpq. The whole cross-compilation thing was brought up repeatedly, e.g. in #61580 (comment).

hoping that we didn't discuss this before and I just didn't find the thread again

Essentially most of the discussion in #61580 is exactly about this. I'd summarize this as: many are for a split, few are for no split. Although, to be fair those numbers are certainly without considering the facts that the outputs are now split and pkgsStatic actually builds....

... that is, it builds on Linux.

One example why libpq is a good idea:

Try building pkgsStatic.libpq and pkgsStatic.postgresql on darwin. libpq succeeds* - and is fairly quick to build, because of fewer dependencies. Building the full postgresql pulls in some other dependencies, takes a lot longer - and then the build eventually fails while building a static perl.**

* which in itself is quite a 🥳 moment for me!
** the fact that it tries to build a static perl is stupid somehow, it shouldn't need that imho. It also tries to build static variants of some build tools like autoconf and automake.. - but the point is still valid: Getting the full postgresql build to work on non-hydra-built-platforms is a lot more complex.

[wolfgangwalther]

FYI pushed a fix for pdo_pgsql/pgsql exts in PHP.

I squashed that into the main libpq commit.

[wolfgangwalther]

Try building pkgsStatic.libpq and pkgsStatic.postgresql on darwin. libpq succeeds - and is fairly quick to build*, because of fewer dependencies. Building the full postgresql pulls in some other dependencies, takes a lot longer - and then the build eventually fails while building a static perl.**

Addition: The same happens for me on pkgsCross.x86_64-freebsd.postgresql vs pkgsCross.x86_64-freebsd.libpq. I can build libpq for freebsd, but not the full postgresql, yet.

[wolfgangwalther]

I'm not sure.
I'm slightly leaning towards this solution

So was I, when I started my comment above. Finding out about darwin and freebsd and the actual practical consequences, I convinced myself. Thus: Merging!

[Ma27]

After sleeping one night over it and reading your comment, I also agree (just ftr).

Thanks a lot for your work here!