Incorrect extra library search path when cross-compiling #4350
Labels
Comments
|
I created a draft PR where I suggest a change. As I do not know all the internals, I may have missed something. I identified 3 ways to fix my issue:
I thought solution 3. was better, as solution 1. seems dirty. As for solution 2., I think the function that parses the script does its job perfectly. I believe it is up to the bit that uses the parsed output to handle the special case. Let me know what you think. In the draft PR, I added a commit (to be removed later obviously) that points to the three parts in the code where I considered making a change. |
Thalyum
pushed a commit
to Thalyum/pyo3
that referenced
this issue
Oct 13, 2024
Thalyum
pushed a commit
to Thalyum/pyo3
that referenced
this issue
Oct 13, 2024
github-merge-queue bot
pushed a commit
that referenced
this issue
Oct 18, 2024
) (#4389) Co-authored-by: Paul-Erwan RIO <[email protected]>
|
Fixed in #4389 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Description
I tried to use PyO3 crate, and cross-compile a simple tool for Aarch64. I was using buildroot to cross-compile for my target:
When I encountered an error during link. Here is the error I got with buildroot:
The error comes from a bad linker option:
-L /usr/lib. Buildroot uses a wrapper around the toolchain (including the linker) that checks if the build tries to use "unsafe" paths, especially in the includes path and library search path. As stated in buildroot documentation:We do not expect cross-compiled library in
/usr/lib, only host libraries or headers. I tried to reproduce with 'manual' cross-compilation. I reused the cross-compiled python interpreter built by Buildroot, as well as its cross-toolchain:setup the linker:
The custom rustflags is to add the search path for libgcc, as well as some verbosity.
Then compile:
RUSTC_LOG=rustc_codegen_ssa::back::link=info PYO3_CROSS=y \ PYO3_PTYHON=<buildroot_path>/output/host/bin/python3.12 \ PYO3_CROSS_LIB_DIR=<buildroot_path>/output/target/usr/lib/ \ cargo build --target=aarch64-unknown-linux-gnuNo issue with manual cross-compilation. Since the linker recognizes the architecture of the libraries in
/usr/liband ignores them as they are for the host:But we still see the same kind of warning.
By using the
PYO3_PRINT_CONFIGvariable, I took a peek at the configuration generated bypyo3-build-config:I created a dummy configuration file with the content above, and I just changed the "lib_dir":
And used it to cross-compile:
RUSTC_LOG=rustc_codegen_ssa::back::link=info PYO3_CROSS=y \ PYO3_PTYHON=<buildroot_path>/output/host/bin/python3.12 \ PYO3_CROSS_LIB_DIR=<buildroot_path>/output/target/usr/lib/ \ PYO3_CONFIG_FILE=$PWD/test.conf \ cargo build --target=aarch64-unknown-linux-gnuAnd the compilation log shows that the 'faulty'
-L /usr/liboption has been replaced by-L /foo/bar:And there is no more warning for the linker.
This configuration file is generated from the
InterpreterConfigstruct, and based on the code comments, "lib_dir" should be the directory where the Python library is located. In the current case, "/usr/lib" is not a correct path.Extra question: since I used
PYO3_CROSS_LIB_DIRenv variable to cross-compile, should not "lib_dir" be filled with the content of this environment variable ?Steps to Reproduce
If you want to reproduce, here is how to use buildroot to get the cross-compiled
python interpreter + cross-toolchain, as well as the simple application code.
simple app code
Here is the code for the tool, a very simple "hello" print:
=> Cargo.toml
=> main.rs
python interpreter cross-compilation with buildroot
git clone https://gitlab.com/buildroot.org/buildroot.git cd buildrootEnable python3 for a basic configuration
Then compile. To be faster, and since only the cross-compilation of python3 is
of interest here, we can compile directly the python3 package:
The build output will be available in
output/targetdirectory. We will find:output/target/usr/bin/python3.12output/target/usr/lib/libpython3.soBuildroot also builds the host version of the same interpreter:
output/host/bin/python3.12Backtrace
No response
Your operating system and version
Linux 6.6.34-1-MANJARO
Your Python version (
python --version)Python 3.12
Your Rust version (
rustc --version)rustc 1.79.0 (129f3b996 2024-06-10)
Your PyO3 version
0.22.0
How did you install python? Did you use a virtualenv?
Host + cross-compiled interpreter using Buildroot, no virtualenv.
Additional Info
No response
The text was updated successfully, but these errors were encountered: