pyo3-build-config: fix cross compilation

[davidhewitt]

This fixes cross-compilation with the new pyo3-build-config crate.

The problem is that CARGO_CFG_TARGET_OS (and similar) environment variables are set to the host OS when running the build.rs in pyo3-build-config, because pyo3-build-config is used in build-dependencies.

The best way I can see to fix this is to evaluate the build configuration in the build.rs of the first runtime dependency we control - PyO3's build.rs.

Before:

• build.rs in pyo3-build-config runs (incorrectly) and generates config information, which gets baked into the pyo3-build-config crate
• downstream crates (pyo3, pyo3-build-config) use this baked-in configuration

This PR:

• build.rs in pyo3-build-config creates an empty OUT_DIR
• build.rs in pyo3 writes the correct config to that OUT_DIR
• downstream crates can load this configuration in their build.rs
• pyo3-macros-backend can load this configuration at macro runtime and check it

cc @birkenfeld this will slightly change how to check for METH_FASTCALL support in #1619 - instead of using cfg!, it's now necessary to use pyo3_build_config::get() and check the configuration directly. This is necessary because pyo3-macros-backend is built before pyo3, so the configuration hasn't yet been written when pyo3-macros-backend is built.

[davidhewitt]

@alonblade I just rebased this branch on your other cross-compilation PR.

Would you be willing to try this branch out to confirm whether it's now fixed the issues with cross-compilation? Thanks.

[alonblade]

@alonblade I just rebased this branch on your other cross-compilation PR.

Would you be willing to try this branch out to confirm whether it's now fixed the issues with cross-compilation? Thanks.

Yep - builds with target armhf on host x86_64.

[messense]

LGTM.

So it's important to note that pyo3-build-config should only be used in build time dependencies for example build.rs or proc macro, never intended to be used in the main pyo3 crate. pyo3 should use normal cfgs.

[davidhewitt]

So it's important to note that pyo3-build-config should only be used in build time dependencies for example build.rs or proc macro, never intended to be used in the main pyo3 crate. pyo3 should use normal cfgs.

Yep precisely. I'm planning to add some updates to Architecture.md to note this.

[davidhewitt]

(I'll merge this after I add some more test coverage.)

[davidhewitt]

I've pushed the coverage much higher. The job is still failing but I'm not sure it's worth doing further (only just slightly under the target), so proceeding to merge.

[alonblade]

Glad you merged it. Btw, somewhat related, I wanted to take the CI for cross builds ticket (I have a setup with bitbucket that works for my pyo3 using project for aarch64 & armhf) but I got stuck on the "I don't grok the yml of github yet" problem. I could just through my current bitbucket CI or even just a docker one using a simplified project, would that help? @davidhewitt

[messense]

@alonblade I've done a lot of cross compilation jobs with GitHub Actions, so I've opened #1700

[alonblade]

@messense sweet.