ci(macos): Avoid linking against Homebrew

[bwoodsend]

Homebrew binaries are always compiled for exactly the version they're installed on making them very unportable. When a wheel is "repaired" by cibuildwheel, delocate-wheel pulls in _psycopg's dependencies (libpq.dylib, libssl.dylib and libcrypto.dylib) which, on a GitHub Actions macOS 14 runner, are provided by Homebrew and are therefore only macOS >= 14 compatible. The resultant wheel is therefore incompatible with all but the latest macOS versions.

Build all dependencies from source so that we can set the deployment target to something sensible. Fixes #1753.

[bwoodsend]

I've temporarily disabled all the other bits of CI just to make verifying quicker. I'll revert all that if/when we get closer to merging. I wanted to gauge whether you think this is worth the trouble before finishing it off? I can't imagine you'd be particularly comfortable with having a rather hairy build step that you can't run locally.

Here's a sample build with the macOS versions in the wheel tags back to normal.

[dvarrazzo]

Yes, this seems useful, and it's what we do for Linux: see the scripts/build/build_libpq.sh. Maybe it would be worth augmenting that script with macOS support? For sure it's better to edit complex bash in a .sh file than interspersed in yaml.

If you complete this operation it would be useful to port it to psycopg 3 too (which has a script pretty much identical).

Hint: you can use if: false to disable a job, which is less invasive then chopping everything away (the reason why there are the if: true in the workflow file is to remember that).

Thank you very much for your initiative!

[dvarrazzo]

No gssapi/ldap support?

[bwoodsend]

Ah, I was going to ask about which --with-xxx features to use but you beat me to it. I see build_libpq.sh makes that decision for me anyway.

[dvarrazzo]

Unfortunately that also implies building these other libraries. Not doing so would break certain features. I don't know how many people rely on them, but I don't feel they are features that we can drop without being unpunished.

TBH, for some reason, GSSAPI causes a lot of problems to macOS users because in some configuration it causes slowdowns on connection, but still it doesn't seem a good case to decide that nobody in the world requires it.

[dvarrazzo]

What is, on macOS, the default socket dir? In other words:

$ psql "host='' port=9999"
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.9999" failed: No such file or directory
	Is the server running locally and accepting connections on that socket?

what path is searched for in macOS? If it's not /tmp then it's worth patching the libpq before building, see

psycopg2/scripts/build/build_libpq.sh

Lines 148 to 152 in e83754a

	# Match the default unix socket dir default with what defined on Ubuntu and
	# Red Hat, which seems the most common location
	sed -i 's|#define DEFAULT_PGSOCKET_DIR .*'\
	'|#define DEFAULT_PGSOCKET_DIR "/var/run/postgresql"|' \
	src/include/pg_config_manual.h

[bwoodsend]

see the scripts/build/build_libpq.sh. Maybe it would be worth augmenting that script with macOS support? For sure it's better to edit complex bash in a .sh file than interspersed in yaml.

Thanks for pointing that out. Yes, adding macOS support to that script sounds better than doing it all over again in YAML.

What is, on macOS, the default socket dir?

I wasn't aware that there was such thing as a socket dir but does this answer your question?

> psql "host='' port=9999"
psql: error: connection to server on socket "/tmp/.s.PGSQL.9999" failed: No such file or directory
	Is the server running locally and accepting connections on that socket?

(That's homebrew's psql – not a source built one in case it matters.)

[bwoodsend]

Do you mind if I bump the OpenLDAP version? I need this fix or the check for POSIX regex throws a false negative. (It's probably sed-able if not.)

[dvarrazzo]

I wasn't aware that there was such thing as a socket dir but does this answer your question?

Yes it does. It seems that the default socket dir is /tmp/, which is the default one in Postgres source, so we don't need patching.

It is useful that it is the brew one. If it wasn't, a program configured and running correctly so far would stop working if we switch to a libpq with a different default. See past issues for example.

Do you mind if I bump the OpenLDAP version?

Absolutely not, go ahead 🙂

[dvarrazzo]

Hello @bwoodsend! How is it going with this work? Is it ready, or are you stuck somewhere, or you don't think you want to finish the MR?

Thank you very much!

[bwoodsend]

Sorry, I'm still here. I ground to a halt over some m4 issue running cyrus-sasl's autoreconf which I couldn't reproduce locally (despite also using macOS 14 M1). I suspect there's something from homebrew on the GitHub runner shadowing the xcode version that works somewhere. When I get a quiet moment where I'm more than half awake, I'll tmate my way in and see if I can figure it out.

[bwoodsend]

Yay! Got there eventually. I'll clean up my mess in a bit and move this out of draft.

I suspect there's something from homebrew on the GitHub runner shadowing the xcode version that works somewhere.

Sure enough, brew install libtool was the magic cure. I never really appreciated before how evil these xcode/brew tools and brew/system libraries dualities were for issue debugging and reproducibility.

The macos-12 GitHub runners have just had the plug pulled on them. I'll bump us onto macos-13 but I'm rather uncomfortable that this time next year, that'll be gone too and we'll have no more x86_64 runners. At that point I suspect we'll be forced to cross compile. I've given it a brief poke locally and it looks like each component insists on doing it in their own way. So not quick and not fun...

[bwoodsend]

Don't suppose you know a better way to force which copy of a library to link against? The build system seems to add homebrew to its search path even if I env -i away all the brew related environment variables.

[dvarrazzo]

Maybe with -L/-l options in the compiler? Not sure.

[bwoodsend]

Up to date verification run: https://github.com/bwoodsend/psycopg2/actions/runs/12291165704

[dvarrazzo]

Hello @bwoodsend

I have worked a bit on this branch, mostly adding libpq build caching and moving the libpq build from the CI workflow to the BEFORE script, consistently with the linux workflow.

A complete package build with all the version run here.

This looks ok to merge for me. Ok for you too?

After this is safe I will also port it to psycopg3, which has the same type of problem.

Thank you very much for your contribution!

[bwoodsend]

Do you want to cherry pick bwoodsend@ede7cd2 in or we can save it for later? I wrote it a while ago but never got around to pushing it (curse my tiny attention span).

I'm happy for this to be merged now either way.

[dvarrazzo]

Do you want to cherry pick bwoodsend@ede7cd2 in or we can save it for later?

If it works, it seems good. I remember that with cross-compiling on macOS you could build but not run/test what built. Is that only a concern for cross-compiling ARM on X86?

---

Edit: answering to myself: looking here it seems that x86 code can be tested on arm.

[bwoodsend]

Do you want to cherry pick bwoodsend@ede7cd2 in or we can save it for later?

If it works, it seems good. I remember that with cross-compiling on macOS you could build but not run/test what built. Is that only a concern for cross-compiling ARM on X86?

Edit: answering to myself: looking here it seems that x86 code can be tested on arm.

Yeah, you can run x86_64 binaries on arm64 using Rosetta emulation but not the other way around. I think that's also scheduled for removal in macOS 16 but we can make the most of it whilst it's still there.

[dvarrazzo]

Thank you for the in-depth co-review, it seems a good improvement to only depend on the ARM runners.

Building packages in this job. If everything works I guess we can merge this.

[bwoodsend]

That's everything I can think of. Ready for the merge button when you are...

[dvarrazzo]

Thank you very much! Merged, now poriting to psycopg 3 in psycopg/psycopg#986.