Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need help with external dependency for einat #16

Closed
vortexilation opened this issue Jun 13, 2024 · 29 comments
Closed

Need help with external dependency for einat #16

vortexilation opened this issue Jun 13, 2024 · 29 comments
Labels
question This is a question

Comments

@vortexilation
Copy link

Describe the bug
I have got problem compiling einat for OpenWrt x86_64 glibc using the muink's einat OpenWrt Package .

At the end of the compilation I am getting rustc cannot find libintl.so.8 :

   Compiling einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)
     Running `rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out`
error: linking with `x86_64-openwrt-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin:/home/user/works/openwrt/staging_dir/hostpkg/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/host/bin:/home/user/works/openwrt/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "x86_64-openwrt-linux-gnu-gcc" "-m64" "/home/user/works/openwrt/tmp/rustcevItVt/symbols.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.build_script_build.900026c213c08482-cgu.0.rcgu.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.2kuwii56gzittnew.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_metadata-77b698b469ccad23.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde_json-d18e1952940c9d6c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libitoa-c99bf860a1243733.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libryu-42733ab1749368f9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libthiserror-aa231274481bae73.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_platform-c20f7cad56b30019.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcamino-a1fd712e042bd596.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libmemmap2-73fca818d05ca4c9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_rs-12af648727bedc4e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libvsprintf-52f5d1afc74d566c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibc-a50be116079b6680.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_sys-6ce4fe8ed806fc57.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libsemver-d9af4a6bbaa40b09.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde-73abfa3b10e52ca3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex-662c0e78adcebbf3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_automata-e8e6e085a55c8ab3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_syntax-f1d457fe75bcbfd6.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libtempfile-e4833a2ad6c4f429.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libfastrand-906c084e5ef6039e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/librustix-4d3b20d2716f8231.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libbitflags-3a495cf5bbfede0b.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblinux_raw_sys-ba02cfb0a81adfba.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcfg_if-1c8851ce38e8a7c1.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libanyhow-0deb8dd3c263e7fc.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18d1c84127d04dc5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-bc53c6e5d5f4111f.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a79bc22e6ae9ee83.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-9c256084533d6dad.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-e53b44d64d10bb35.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-9d0b3274a5b15e82.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-b0fb619b58a6601a.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-1319f451087d6793.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e7ba5787d3fb181b.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-e23167ed65317fed.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-52af0786d05d61a5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-28f9d348427e8d43.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-689892118ddcc60e.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-a1f86f0be6e775e6.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-cbc93ea126f55c98.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-390f90785cbfdde3.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-aded7817e4ea9283.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6c16e1294e732b84.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-a8a94ae78107d089.rlib" "-Wl,-Bdynamic" "-lelf" "-lz" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: warning: libintl.so.8, needed by /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so: undefined reference to `libintl_dgettext'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `einat` (build script) due to 1 previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
error: failed to compile `einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)`, intermediate artifacts can be found at `/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

the libintl.so.8 is located in :

/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib

I am not a rust developer by any means but after doing some research, I have modified the build.rs / build script as the following :

fn main() {
    let mut out =
        PathBuf::from(env::var_os("OUT_DIR").expect("OUT_DIR must be set in build script"));
    out.push("einat.skel.rs");

    let mut c_args = vec![
        "-Wno-compare-distinct-pointer-types".to_string(),
        "-mcpu=v3".to_string(),
    ];

    if cfg!(feature = "ipv6") {
        c_args.push("-DFEAT_IPV6".to_string());
    }

    SkeletonBuilder::new()
        .source(SRC)
        .clang_args(c_args)
        .debug(true)
        .build_and_generate(&out)
        .unwrap();
    println!("cargo:rustc-link-lib=dylib=libintl.so.8");
    println!("cargo:rustc-link-search=native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib");
    println!("cargo:rustc-link-arg=-Wl,-rpath,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib");
    println!("cargo:rerun-if-changed={SRC}");
}

But doesn't get any differences.

Expected behavior
Able to compile einat for OpenWrt x86_64 with CONFIG_BUILD_NLS enabled.

Target Machine (please complete the following information):

  • Architecture: x86-64
  • Linux distribution: OpenWrt
  • Kernel version: 6.6.32
  • einat version: 0.1.2
@EHfive
Copy link
Owner

EHfive commented Jun 13, 2024

Saw your post openwrt/packages#24375 from your profile, it's the libelf.so on your system failed to link with libintl.so.

root@buildsystem:/home/user/works/openwrt# ldd /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so
        linux-vdso.so.1 (0x00007ffd965f1000)
        libintl.so.8 => not found
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f146c828000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f146c7fb000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f146c616000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f146c872000)

This is less on einat as we don't build libelf.so but merely link to it using ld, and we don't directly link to libintl.so or use it in any way. And any other user of libelf.so on your build system would encounter the same problem.

The error pops up because ld cannot found libintl.so on your system in the first place. This is likely a misconfiguration of ld search path of libintl.so with CONFIG_BUILD_NLS enabled or missing of rpath to libintl.so on libelf.so, as the libintl.so does not located in standard location.

Because my OpenWrt build is NLS enabled (CONFIG_BUILD_NLS), libintl.so.8 is located in here :
/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib

I am not sure why your LD_LIBRARY_PATH did not picked by ld or ld-linux.so, maybe the path was wrong? Make sure lld on libelf.so can also find libintl.so.

For passing library search path to rustc then to ld, you can use environment variable CARGO_TARGET_<RUST_TARGET>_RUSTFLAGS, the RUSTFLAGS would not be used for building build script if Cargo --target is specified. Though this is a hack way and should not be preferred. Also I am not sure this can work as library search path passed this way would not affect ld-linux.so.

export CARGO_TARGET_x86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-L/path/to/dir/of/libintl/lib"

I am not familiar with OpenWrt build system, maybe @muink can give some insight.

@EHfive EHfive added the question This is a question label Jun 13, 2024
@vortexilation
Copy link
Author

This is less on einat as we don't build libelf.so but merely link to it using ld, and we don't directly link to libintl.so or use it in any way. And any other user of libelf.so on your build system would encounter the same problem.

Any others OpenWrt package that are using libelf seems to be unaffected.

The error pops up because ld cannot found libintl.so on your system in the first place. This is likely a misconfiguration of ld search path of libintl.so with CONFIG_BUILD_NLS enabled or missing of rpath to libintl.so on libelf.so, as the libintl.so does not located in standard location.

Any attempt to make OpenWrt rustc toolchain to notice rpath or rpath-link seems failed in my tries. Seems like rustc just ignore whatever my variables throw at it.

I am not sure why your LD_LIBRARY_PATH did not picked by ld or ld-linux.so, maybe the path was wrong? Make sure lld on libelf.so can also find libintl.so.

If I am doing at shell not OpenWrt build system :

export LD_LIBRARY_PATH=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so

before ldd then it can found libintl.so.8 but compilation still failing.

Your proposes workaround also weirdly not picked up by rustc, still the same error.

export CARGO_TARGET_x86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-L/path/to/dir/of/libintl/lib"

@muink
Copy link

muink commented Jun 14, 2024

I'm not familiar with rust build system. The following code reveals the binary build process of a rust application in openwrt and may provide some reference.
https://github.com/openwrt/packages/blob/dd423c5936aba8d4c512cf74d65c30514d16bc5c/lang/rust/rust-package.mk#L25-L34

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

Any others OpenWrt package that are using libelf seems to be unaffected.

I mean the build system specifically, as ld can't find libintl.so in that environment. If other package linking to libelf can build successfully, they must have some kind of extra setup in their build process which you might want to apply too.

Any attempt to make OpenWrt rustc toolchain to notice rpath or rpath-link seems failed in my tries. Seems like rustc just ignore whatever my variables throw at it.

It's about ld not rustc, rustc uses ld to link things. Also as I said previously einat does not link to libintl.so but only its dependent libelf.so, if your system has broken libelf.so, it the problem of the system.

As a workaround, try adding a rapth to libelf.so (e.g. using patchelf) so it can find libintl.so. Also I am wondering what is the rpath of your libelf.so.

If I am doing at shell not OpenWrt build system :

export LD_LIBRARY_PATH=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so

This should be path to parent directory of the library, not path to the library itself. Also you should fill path to parent directory of libintl.so.

As you are doing this externally, it could be that OpenWrt build system overrides LD_LIBRARY_PATH.


I have noticed it's the build script of einat failed to link with libelf.so, both the einat running on host system and its build script running on build system needs libelf.so in their own running system respectively.

However in build process ld is linking the build script to libelf.so from host system. Given https://github.com/muink/openwrt-einat-ebpf#build suggests to install libelf-dev and zlib1g-dev on your system, maybe the build script should link to libelf from external build system instead? @muink Do you think this is right?

@vortexilation
Copy link
Author

I mean the build system specifically, as ld can't find libintl.so in that environment. If other package linking to libelf can build successfully, they must have some kind of extra setup in their build process which you might want to apply too.

In OpenWrt we use somekind of OpenWrt package Makefile to make to package compilation going on, in comparison with other package, I don't see anything differences on muink's einat OpenWrt package Makefile with other rust app that are compiled on OpenWrt. Probably this is between OpenWrt inner build process or bugs in rustc itself?.

It's about ld not rustc, rustc uses ld to link things. Also as I said previously einat does not link to libintl.so but only its dependent libelf.so, if your system has broken libelf.so, it the problem of the system.

I think the libelf that are used are the one from OpenWrt :

  = note: /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: warning: libintl.so.8, needed by /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so: undefined reference to `libintl_dgettext'

This should be path to parent directory of the library, not path to the library itself. Also you should fill path to parent directory of libintl.so.

It's a typo, sorry.

As you are doing this externally, it could be that OpenWrt build system overrides LD_LIBRARY_PATH.

I have tried to override LD_LIBRARY_PATH from OpenWrt also getting ignored.

However in build process ld is linking the build script to libelf.so from host system. Given https://github.com/muink/openwrt-einat-ebpf#build suggests to install libelf-dev and zlib1g-dev on your system, maybe the build script should link to libelf from external build system instead? @muink Do you think this is right?

Both of libelf-dev and zlib1g-dev are already installed.

@vortexilation
Copy link
Author

I'm not familiar with rust build system. The following code reveals the binary build process of a rust application in openwrt and may provide some reference.

Already checked that out and tried some tinkering on it but still no progress. So you don't enabled CONFIG_BUILD_NLS ?.

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

I have pushed 3174aca to branch pre-gen-skel, this commit drops libelf and zlib requirements on build platform(the host platform still needs them). Try building openwrt-einat-ebpf with this branch.

If this works for you, than presumably it's a problem on setup of build platform dependencies in OpenWrt's Rust build process. As the build script of einat is supposed to link against libelf on build platform.

@muink
Copy link

muink commented Jun 14, 2024

Already checked that out and tried some tinkering on it but still no progress. So you don't enabled CONFIG_BUILD_NLS ?.

Since the compilation machine is not at hand at the moment I will test it at a later time.

However in build process ld is linking the build script to libelf.so from host system. Given https://github.com/muink/openwrt-einat-ebpf#build suggests to install libelf-dev and zlib1g-dev on your system, maybe the build script should link to libelf from external build system instead? @muink Do you think this is right?

I think the libelf that are used are the one from OpenWrt :

Building openwrt-einat-ebpf itself does not require these two packages, but the host toolchain used to build it seems to.
And the official buildworker will also install them
https://github.com/openwrt/buildbot/blob/main/docker/buildworker/Dockerfile#L16

Normally openwrt-einat-ebpf gets libelf.so from the following path

$TOPDIR/staging_dir/host/lib/libelf.so
$TOPDIR/staging_dir/target-x86_64_musl/usr/lib/libelf.so

They are built by the host tool elfutils
Host https://github.com/openwrt/openwrt/blob/main/tools/elfutils/Makefile
Target https://github.com/openwrt/openwrt/blob/main/package/libs/elfutils/Makefile

@vortexilation
Copy link
Author

vortexilation commented Jun 14, 2024

I have pushed 3174aca to branch pre-gen-skel, this commit drops libelf and zlib requirements on build platform(the host platform still needs them). Try building openwrt-einat-ebpf with this branch.

Thank you for libelf workaround but still the same errors, although it now took quite more longer than before to compiling and reach that error.

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

Ok, then it seems like the problem is on target libelf. The ld for build script yells because the build platform and the host platform has the same architecture x86-64 so it looks for broken libelf for host platform by some order.

Also seems @muink's build of openwrt-einat-ebpf uses musl instead of glibc, not sure if this is related or not.

PS: I am describing build platform and host platform from perspective of target machine. They may also be called as host and target respectively from perspective of build machine.

@vortexilation
Copy link
Author

I have patchelf the libelf itself, now ldd result (even without setting LD_LIBRARY_PATH):

root@buildsystem:/home/user/works/openwrt# ldd /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so
        linux-vdso.so.1 (0x00007ffc7c74d000)
        libintl.so.8 => /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib/libintl.so.8 (0x00007f7032658000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f7032631000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7032604000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f703241f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f7032697000)

But compilation still failing.

Yes I am using glibc targeting x86_64.

rustc seems to ignore we everything throw at it, I really wonder why. I am sure it's as simple as rustc is accepting rpath or rpath-link.

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

What is the log after your added rpath to libelf? Did you use branch pre-gen-skel or not?

Also try using "-C link-args" on CARGO_TARGET_<RUST_TARGET>_RUSTFLAGS mentioned above. "cargo:rustc-link-arg=" in einat's build script won't affect itself.

@muink
Copy link

muink commented Jun 14, 2024

x86_64 Ubuntu 22.04 openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64

sudo apt install build-essential ccache clang curl flex bison g++ gawk \
gcc-multilib g++-multilib genisoimage gettext git libdw-dev libelf-dev \
libncurses5-dev libssl-dev locales pv pwgen python3 python3-pip \
python3-setuptools qemu-utils rsync signify-openbsd subversion swig unzip \
zlib1g-dev file wget zstd

CONFIG_BUILD_NLS=y

The same build failed

make[2]: Entering directory '/media/gp/opby/fantastic_packages/packages/openwrt-einat-ebpf'
rm -f /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/.built
touch /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/.built_check
CARGO_BUILD_TARGET=x86_64-unknown-linux-musl CARGO_HOME=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/dl/cargo CARGO_PROFILE_RELEASE_CODEGEN_UNITS=1 CARGO_PROFILE_RELEASE_DEBUG=false CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS=false CARGO_PROFILE_RELEASE_LTO=true CARGO_PROFILE_RELEASE_OPT_LEVEL=z CARGO_PROFILE_RELEASE_OVERFLOW_CHECKS=true CARGO_PROFILE_RELEASE_PANIC=unwind CARGO_PROFILE_RELEASE_RPATH=false CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-openwrt-linux-musl-gcc RUSTFLAGS="-Ctarget-feature=-crt-static -lssp_nonshared" TARGET_CC=x86_64-openwrt-linux-musl-gcc TARGET_CFLAGS="-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -ffile-prefix-map=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2=einat-ebpf-0.1.2 -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro " CC=gcc MAKEFLAGS="" cargo install -v --profile release  --root /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/ipkg-install --path "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/" -j1 
  Installing einat v0.1.2 (/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2)
    Updating crates.io index
       Fresh unicode-ident v1.0.12
       Fresh proc-macro2 v1.0.85
       Fresh quote v1.0.36
       Fresh syn v2.0.66
       Fresh serde_derive v1.0.203
       Fresh cc v1.0.99
       Fresh cfg_aliases v0.1.1
       Fresh autocfg v1.3.0
       Fresh libc v0.2.155
       Fresh nix v0.28.0
       Fresh thiserror-impl v1.0.61
       Fresh pkg-config v0.3.30
       Fresh pin-project-lite v0.2.14
       Fresh futures-core v0.3.30
       Fresh utf8parse v0.2.2
       Fresh futures-sink v0.3.30
       Fresh futures-channel v0.3.30
       Fresh anstyle-parse v0.2.4
       Fresh serde v1.0.203
       Fresh slab v0.4.9
       Fresh rustversion v1.0.17
       Fresh syn v1.0.109
       Fresh futures-macro v0.3.30
       Fresh anstyle v1.0.7
       Fresh memchr v2.7.4
       Fresh colorchoice v1.0.1
       Fresh is_terminal_polyfill v1.70.0
       Fresh pin-utils v0.1.0
       Fresh anstyle-query v1.1.0
       Fresh heck v0.4.1
       Fresh futures-io v0.3.30
       Fresh futures-task v0.3.30
       Fresh futures-util v0.3.30
       Fresh strum_macros v0.24.3
       Fresh anstream v0.6.14
       Fresh paste v1.0.15
       Fresh bytemuck_derive v1.7.0
       Fresh regex-syntax v0.8.4
       Fresh strsim v0.11.1
       Fresh clap_lex v0.7.1
       Fresh ryu v1.0.18
       Fresh heck v0.5.0
       Fresh byteorder v1.5.0
       Fresh itoa v1.0.11
       Fresh linux-raw-sys v0.4.14
       Fresh rustix v0.38.34
       Fresh serde_json v1.0.117
       Fresh netlink-packet-utils v0.5.2
       Fresh clap_derive v4.5.5
       Fresh clap_builder v4.5.7
       Fresh regex-automata v0.4.7
       Fresh bytemuck v1.16.0
       Fresh thiserror v1.0.61
       Fresh semver v1.0.23
       Fresh camino v1.1.7
       Fresh futures-executor v0.3.30
       Fresh cargo-platform v0.1.8
       Fresh socket2 v0.5.7
       Fresh signal-hook-registry v1.4.2
       Fresh mio v0.8.11
       Fresh tokio-macros v2.3.0
       Fresh bytes v1.6.0
       Fresh log v0.4.21
       Fresh once_cell v1.19.0
       Fresh cfg-if v1.0.0
       Fresh fastrand v2.1.0
       Fresh tempfile v3.10.1
       Fresh tokio v1.38.0
       Fresh anyhow v1.0.86
       Fresh cargo_metadata v0.15.4
       Fresh futures v0.3.30
       Fresh bitflags v2.5.0
       Fresh regex v1.10.5
       Fresh clap v4.5.7
       Fresh netlink-packet-core v0.7.0
       Fresh memmap2 v0.5.10
       Fresh equivalent v1.0.1
       Fresh hashbrown v0.14.5
       Fresh indexmap v2.2.6
       Fresh libbpf-cargo v0.23.3
       Fresh netlink-sys v0.8.6
       Fresh tracing-core v0.1.32
       Fresh toml_datetime v0.6.6
       Fresh serde_spanned v0.6.6
       Fresh winnow v0.6.13
       Fresh lazy_static v1.4.0
       Fresh overload v0.1.1
       Fresh nu-ansi-term v0.46.0
       Fresh sharded-slab v0.1.7
       Fresh toml_edit v0.22.14
       Fresh vsprintf v2.0.0
       Fresh netlink-proto v0.11.3
       Fresh num-traits v0.2.19
       Fresh netlink-packet-route v0.19.0
       Fresh nix v0.27.1
       Fresh libbpf-sys v1.4.2+v1.4.2
       Fresh thread_local v1.1.8
       Fresh ipnet v2.9.0
       Fresh async-stream-impl v0.3.5
       Fresh fundu-core v0.3.0
       Fresh fundu v2.0.0
       Fresh async-stream v0.3.5
       Fresh prefix-trie v0.3.0
       Fresh tracing-subscriber v0.3.18
       Fresh libbpf-rs v0.23.3
       Fresh rtnetlink v0.14.1
       Fresh toml v0.8.14
       Fresh tracing v0.1.40
       Fresh lexopt v0.3.0
   Compiling einat v0.1.2 (/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2)
     Running `rustc --crate-name einat --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=z -C lto -C codegen-units=1 -C overflow-checks=on --cfg 'feature="default"' -C metadata=0528bfb27c50767c -C extra-filename=-0528bfb27c50767c --out-dir /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -C linker=x86_64-openwrt-linux-musl-gcc -C strip=symbols -L dependency=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps -L dependency=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release/deps --extern anyhow=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libanyhow-1cc338482140e58f.rlib --extern async_stream=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libasync_stream-61fdc28b774102b6.rlib --extern bitflags=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libbitflags-cdef05a88efaf31e.rlib --extern bytemuck=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libbytemuck-2684d09a4c0e1b72.rlib --extern fundu=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libfundu-91ceebfe201ed0a8.rlib --extern futures_util=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libfutures_util-749240121ad1eb7f.rlib --extern ipnet=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libipnet-68e6a1300806c38c.rlib --extern lexopt=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblexopt-e3acaf9751def6ae.rlib --extern libbpf_rs=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblibbpf_rs-6e20585a623eb06e.rlib --extern libbpf_sys=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblibbpf_sys-ca95b2d3657cd827.rlib --extern libc=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblibc-d4ad94795f353415.rlib --extern netlink_packet_core=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libnetlink_packet_core-7874534a7e7e0b07.rlib --extern netlink_packet_route=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libnetlink_packet_route-7aebcd096c0c2dd9.rlib --extern netlink_sys=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libnetlink_sys-b497d7a76611672b.rlib --extern prefix_trie=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libprefix_trie-97da8a16023722c0.rlib --extern rtnetlink=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/librtnetlink-0b4dfa530c9bc9d3.rlib --extern serde=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libserde-d174f0ad3c105fe5.rlib --extern tokio=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtokio-3e61cd4369bdcece.rlib --extern toml=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtoml-06cb3ce3e67ce19f.rlib --extern tracing=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtracing-661f903d80c4a3cd.rlib --extern tracing_subscriber=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtracing_subscriber-e989e7edea0f4cea.rlib -Ctarget-feature=-crt-static -lssp_nonshared -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/build/libbpf-sys-db4f1037fb33e73b/out -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release/deps -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/build/vsprintf-afb5d085347053aa/out`
error: linking with `x86_64-openwrt-linux-musl-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-gnu/bin:/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/bin:/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/hostpkg/bin:/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/bin:/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/bin:/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/host/bin:/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin" VSLANG="1033" "x86_64-openwrt-linux-musl-gcc" "-m64" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/tmp/rustcUzVXPc/symbols.o" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/einat-0528bfb27c50767c.einat.409e59d788da03a7-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release/deps" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/build/libbpf-sys-db4f1037fb33e73b/out" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release/deps" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/build/vsprintf-afb5d085347053aa/out" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,-Bdynamic" "-lssp_nonshared" "-Wl,-Bstatic" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/tmp/rustcUzVXPc/libvsprintf-8e3c258e78583855.rlib" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/tmp/rustcUzVXPc/liblibbpf_sys-ca95b2d3657cd827.rlib" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-00cb08fa6cd7956c.rlib" "-Wl,-Bdynamic" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lelf" "-lz" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lssp_nonshared" "-lgcc_s" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-musl/lib" "-o" "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/einat-0528bfb27c50767c" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/bin/../lib/gcc/x86_64-openwrt-linux-musl/12.3.0/../../../../x86_64-openwrt-linux-musl/bin/ld: warning: libintl.so.8, needed by /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/toolchain-x86_64_gcc-12.3.0_musl/bin/../lib/gcc/x86_64-openwrt-linux-musl/12.3.0/../../../../x86_64-openwrt-linux-musl/bin/ld: /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/usr/lib/libelf.so: undefined reference to `libintl_dgettext'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `einat` (bin "einat") due to previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name einat --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=z -C lto -C codegen-units=1 -C overflow-checks=on --cfg 'feature="default"' -C metadata=0528bfb27c50767c -C extra-filename=-0528bfb27c50767c --out-dir /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps --target x86_64-unknown-linux-musl -C linker=x86_64-openwrt-linux-musl-gcc -C strip=symbols -L dependency=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps -L dependency=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release/deps --extern anyhow=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libanyhow-1cc338482140e58f.rlib --extern async_stream=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libasync_stream-61fdc28b774102b6.rlib --extern bitflags=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libbitflags-cdef05a88efaf31e.rlib --extern bytemuck=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libbytemuck-2684d09a4c0e1b72.rlib --extern fundu=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libfundu-91ceebfe201ed0a8.rlib --extern futures_util=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libfutures_util-749240121ad1eb7f.rlib --extern ipnet=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libipnet-68e6a1300806c38c.rlib --extern lexopt=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblexopt-e3acaf9751def6ae.rlib --extern libbpf_rs=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblibbpf_rs-6e20585a623eb06e.rlib --extern libbpf_sys=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblibbpf_sys-ca95b2d3657cd827.rlib --extern libc=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/liblibc-d4ad94795f353415.rlib --extern netlink_packet_core=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libnetlink_packet_core-7874534a7e7e0b07.rlib --extern netlink_packet_route=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libnetlink_packet_route-7aebcd096c0c2dd9.rlib --extern netlink_sys=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libnetlink_sys-b497d7a76611672b.rlib --extern prefix_trie=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libprefix_trie-97da8a16023722c0.rlib --extern rtnetlink=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/librtnetlink-0b4dfa530c9bc9d3.rlib --extern serde=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libserde-d174f0ad3c105fe5.rlib --extern tokio=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtokio-3e61cd4369bdcece.rlib --extern toml=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtoml-06cb3ce3e67ce19f.rlib --extern tracing=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtracing-661f903d80c4a3cd.rlib --extern tracing_subscriber=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/deps/libtracing_subscriber-e989e7edea0f4cea.rlib -Ctarget-feature=-crt-static -lssp_nonshared -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/build/libbpf-sys-db4f1037fb33e73b/out -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release/deps -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/release -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target/x86_64-unknown-linux-musl/release/build/vsprintf-afb5d085347053aa/out` (exit status: 1)
error: failed to compile `einat v0.1.2 (/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2)`, intermediate artifacts can be found at `/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
make[2]: *** [Makefile:84: /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/.built] Error 101
make[2]: Leaving directory '/media/gp/opby/fantastic_packages/packages/openwrt-einat-ebpf'
time: package/feeds/fantastic_packages_special/einat-ebpf/compile#14.94#0.38#16.47
    ERROR: package/feeds/fantastic_packages_special/einat-ebpf failed to build.
make[1]: *** [package/Makefile:129: package/feeds/fantastic_packages_special/einat-ebpf/compile] Error 1
make[1]: Leaving directory '/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64'
make: *** [/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/include/toplevel.mk:225:package/einat-ebpf/compile] Error 2

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

There is no rpath flags in ld command arguments. So you switched to musl, the <RUST_TARGET> needs to be adjusted accordingly.

CARGO_TARGET_x86_64_UNKNOWN_LINUX_MUSL_RUSTFLAGS

CARGO_BUILD_TARGET=x86_64-unknown-linux-musl
error: linking with x86_64-openwrt-linux-musl-gcc failed: exit status: 1

Also have you not patched libelf.so or it's been overridden during build? Otherwise ld should not fails like that.

Sorry, looking at wrong person.

@vortexilation
Copy link
Author

vortexilation commented Jun 14, 2024

What is the log after your added rpath to libelf? Did you use branch pre-gen-skel or not?

The same errors log, I think identical, now trying to pass very verbose to cargo. Yes I am using pre-gen-skel

Also try using "-C link-args" on CARGO_TARGET_<RUST_TARGET>_RUSTFLAGS mentioned above. "cargo:rustc-link-arg=" in einat's build script won't affect itself.

like these ? :

export CARGO_TARGET_x86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-args=-Wl,-rpath,$(STAGING_DIR)/usr/lib/libintl-full/lib"

@muink
Interesting, so musl also is affected.

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

like these ? :

export CARGO_TARGET_x86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS="-C link-args=-Wl,-rpath,$(STAGING_DIR)/usr/lib/libintl-full/lib"

Yes, maybe you can also try append --unresolved-symbols=ignore-all or --unresolved-symbols=ignore-in-shared-libs(see man 1 ld) to ignore symbol missing error.

edit: report-all -> ignore-all

@vortexilation
Copy link
Author

Yes, maybe you can also try append --unresolved-symbols=report-all or --unresolved-symbols=ignore-in-shared-libs(see man 1 ld) to ignore symbol missing error.

In where do i need to append it?, can you give an example?.

cargo very verbose (-vv) log :

   Compiling einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)
     Running `CARGO=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3 CARGO_PKG_AUTHORS='Huang-Huang Bao <[email protected]>' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=GPL-2.0 CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=einat CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/EHfive/einat-ebpf' CARGO_PKG_RUST_VERSION=1.74 CARGO_PKG_VERSION=0.1.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out`
warning: `toml` (lib) generated 1 warning
error: linking with `x86_64-openwrt-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin:/home/user/works/openwrt/staging_dir/hostpkg/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/bin:/home/user/works/openwrt/staging_dir/host/bin:/home/user/works/openwrt/staging_dir/host/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "x86_64-openwrt-linux-gnu-gcc" "-m64" "/home/user/works/openwrt/tmp/rustck84dLP/symbols.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.build_script_build.900026c213c08482-cgu.0.rcgu.o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec.2kuwii56gzittnew.rcgu.o" "-Wl,--as-needed" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_metadata-77b698b469ccad23.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde_json-d18e1952940c9d6c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libitoa-c99bf860a1243733.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libryu-42733ab1749368f9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libthiserror-aa231274481bae73.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcargo_platform-c20f7cad56b30019.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcamino-a1fd712e042bd596.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libmemmap2-73fca818d05ca4c9.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_rs-12af648727bedc4e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libvsprintf-52f5d1afc74d566c.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibc-a50be116079b6680.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_sys-6ce4fe8ed806fc57.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libsemver-d9af4a6bbaa40b09.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libserde-73abfa3b10e52ca3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex-662c0e78adcebbf3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_automata-e8e6e085a55c8ab3.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libregex_syntax-f1d457fe75bcbfd6.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libtempfile-e4833a2ad6c4f429.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libfastrand-906c084e5ef6039e.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/librustix-4d3b20d2716f8231.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libbitflags-3a495cf5bbfede0b.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblinux_raw_sys-ba02cfb0a81adfba.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libcfg_if-1c8851ce38e8a7c1.rlib" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/libanyhow-0deb8dd3c263e7fc.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-18d1c84127d04dc5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-bc53c6e5d5f4111f.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a79bc22e6ae9ee83.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-9c256084533d6dad.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-e53b44d64d10bb35.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-9d0b3274a5b15e82.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-b0fb619b58a6601a.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-1319f451087d6793.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e7ba5787d3fb181b.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-e23167ed65317fed.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-52af0786d05d61a5.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-28f9d348427e8d43.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-689892118ddcc60e.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-a1f86f0be6e775e6.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-cbc93ea126f55c98.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-390f90785cbfdde3.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-aded7817e4ea9283.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6c16e1294e732b84.rlib" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-a8a94ae78107d089.rlib" "-Wl,-Bdynamic" "-lelf" "-lz" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec/build_script_build-13037c25982734ec" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-all" "-nodefaultlibs"
  = note: /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: warning: libintl.so.8, needed by /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so, not found (try using -rpath or -rpath-link)
          /home/user/works/openwrt/staging_dir/toolchain-x86_64_gcc-14.1.0_glibc/lib/gcc/x86_64-openwrt-linux-gnu/14.1.0/../../../../x86_64-openwrt-linux-gnu/bin/ld: /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf.so: undefined reference to `libintl_dgettext'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `einat` (build script) due to 1 previous error

Caused by:
  process didn't exit successfully: `CARGO=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3 CARGO_PKG_AUTHORS='Huang-Huang Bao <[email protected]>' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=GPL-2.0 CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=einat CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/EHfive/einat-ebpf' CARGO_PKG_RUST_VERSION=1.74 CARGO_PKG_VERSION=0.1.2 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=2 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps rustc --crate-name build_script_build --edition=2021 build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="default"' --cfg 'feature="ipv6"' -C metadata=13037c25982734ec -C extra-filename=-13037c25982734ec --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/einat-13037c25982734ec -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --extern libbpf_cargo=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps/liblibbpf_cargo-fc7173ecf7072a22.rlib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/libbpf-sys-de20753a55c71e06/out -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release -L native=/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/host/lib/rustlib/x86_64-unknown-linux-gnu/lib -L native=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
warning: `toml_edit` (lib) generated 12 warnings
error: failed to compile `einat v0.1.2 (/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3)`, intermediate artifacts can be found at `/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
make[2]: *** [Makefile:92: /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/.built] Error 101
make[2]: Leaving directory '/home/user/works/openwrt/feeds/packages/net/einat'
time: package/feeds/packages/einat/compile#137.74#13.65#30.42
    ERROR: package/feeds/packages/einat failed to build.
make[1]: *** [package/Makefile:173: package/feeds/packages/einat/compile] Error 1
make[1]: Leaving directory '/home/user/works/openwrt'
make: *** [/home/user/works/openwrt/include/toplevel.mk:233: package/einat/compile] Error 2

@muink
Thanks for joining in, I hope that we can tackle this issue.

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

In where do i need to append it?, can you give an example?.

It's a linker flag, this is same as how you supply the -rpath to gcc with -Wl,.

Seems you don't understand basics on how linkage works..

@vortexilation
Copy link
Author

vortexilation commented Jun 14, 2024

I quite understand the basic but not in-depth, imho isn't it we passing -W1 to gcc with CFLAGS? .

Isn't it in rustc with RUSTFLAGS env variable?

[EDIT]
So it's LDFLAGS ?

@muink
Copy link

muink commented Jun 14, 2024

CARGO_BUILD_TARGET=x86_64-unknown-linux-musl 
CARGO_HOME=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/dl/cargo 
CARGO_PROFILE_RELEASE_CODEGEN_UNITS=1 
CARGO_PROFILE_RELEASE_DEBUG=false 
CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS=false 
CARGO_PROFILE_RELEASE_LTO=true 
CARGO_PROFILE_RELEASE_OPT_LEVEL=z 
CARGO_PROFILE_RELEASE_OVERFLOW_CHECKS=true 
CARGO_PROFILE_RELEASE_PANIC=unwind 
CARGO_PROFILE_RELEASE_RPATH=false 
CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-openwrt-linux-musl-gcc 
RUSTFLAGS="-Ctarget-feature=-crt-static -L/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/staging_dir/target-x86_64_musl/usr/lib/libintl-full/lib -lssp_nonshared" 
TARGET_CC=x86_64-openwrt-linux-musl-gcc 
TARGET_CFLAGS="-Os -pipe -fno-caller-saves -fno-plt -fhonour-copts -ffile-prefix-map=/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2=einat-ebpf-0.1.2 -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro " 
CC=gcc MAKEFLAGS="" 
cargo install -v --profile release  --root /media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/ipkg-install --path "/media/gp/opby/openwrt-sdk-23.05.3-x86-64_gcc-12.3.0_musl.Linux-x86_64/build_dir/target-x86_64_musl/einat-ebpf-0.1.2/" -j1 

I tried appending -L$(STAGING_DIR)/usr/lib/libintl-full/lib for RUSTFLAGS, but it seems to have no effect

@vortexilation
Copy link
Author

It's like rustc just ignoring it?, that's probably what I've found also.

@EHfive
Copy link
Owner

EHfive commented Jun 14, 2024

It's like rustc just ignoring it?, that's probably what I've found also.

For your case, it's not rustc ignoring the RUSTFLAGS, (1) it's just the content in *RUSTFLAGS has never been passed to cargo/rustc. The proof is that "-L" flag or "-C link-args=.." you intended to add was never shown in "Running `CARGO=..." line in your log.

Please stop being misleading for implying it's the rustc causes the problem. The issue here is that indirectly referenced library libintl.so can not be find by ld hence the error due to its default --unresolved-symbols=report-all policy. This was never a concern of rustc.

I believe if we can pass -Wl,--unresolved-symbols=ignore-all or -Wl,--unresolved-symbols=ignore-in-shared-libs trough *RUSTFLAGS="-C link-args" to gcc thus to ld that rustc uses for linking , the einat or it's build script binary can be built successfully.

# XXX: Figure out how this can actually be passed to cargo.
CARGO_TARGET_<RUST_TARGET>_RUSTFLAGS="-C link-args=-Wl,--unresolved-symbols=ignore-all"

However the runtime exection of einat or it's build script binary is not guaranteed as ld-linux.so(do not confuse this with ld) can not find libintl.so used by libelf.so.

(2) For @vortexilation 's case, the build script is linking with potentially broken libelf from host platform instead of libelf from build platform. So if not using pre-gen-skel branch, even the build script binary can be built successfully it can still not be executed on build platform, thus the build of einat binary would still fail.

(3) What need to be resolved first here is why neither ld or ld-linux.so can not locate libintl.so from libelf.so. Could it be LD_LIBRARY_PATH can not been seen by ld or been overridden in the middle of build? Can libintl.so be located from libelf.so by ld-linux.so on running target machine. Your patchelf does not work either because the file was rebuild after modification or modify a wrong file.

(1) (2) (3) are separate issues that need to be resolved.

@muink
Copy link

muink commented Jun 14, 2024

muink/openwrt-einat-ebpf@fdfb646 The RUSTFLAGS based on ignore-in-shared-libs are build passed.
But RUSTFLAGS+= -C link-args=-L$(STAGING_DIR)/usr/lib/libintl-full/lib -Wl,-rpath-link=$(STAGING_DIR)/usr/lib/libintl-full/lib is not.

@vortexilation
Copy link
Author

  1. About patchelf, even after few einat compilation attempt, the ldd result still able to resolve or return libintl.so.8 (no LD_LIBRARY_PATH env var being set)
root@buildsystem:~# ldd /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libelf-0.191.so 
        linux-vdso.so.1 (0x00007ffc2cc88000)
        libintl.so.8 => /home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib/libintl.so.8 (0x00007fd428bd1000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd428baa000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd428b7d000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd428998000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd428c10000)

@vortexilation
Copy link
Author

vortexilation commented Jun 15, 2024

Managed to somehow passing into LD but not parse-able by LD/gcc itself ? :

CFLAGS = "-Wl,-rpath-link,$(STAGING_DIR)/usr/lib/libintl-full/lib"
     Running `rustc --crate-name linux_raw_sys --edition=2021 /home/user/works/openwrt/dl/cargo/registry/src/index.crates.io-6f17d22bba15001f/linux-raw-sys-0.4.14/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C overflow-checks=on --cfg 'feature="elf"' --cfg 'feature="errno"' --cfg 'feature="general"' --cfg 'feature="ioctl"' --cfg 'feature="no_std"' -C metadata=ba02cfb0a81adfba -C extra-filename=-ba02cfb0a81adfba --out-dir /home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps -C linker=x86_64-openwrt-linux-gnu-gcc -C strip=symbols -L dependency=/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/deps --cap-lints allow`
The following warnings were emitted during compilation:

warning: [email protected]: gcc: warning: "-Wl,-rpath-link,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib": linker input file unused because linking not done
warning: [email protected]: gcc: error: "-Wl,-rpath-link,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib": linker input file not found: No such file or directory

error: failed to run custom build command for `vsprintf v2.0.0`

Caused by:
  process didn't exit successfully: `/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-b22cfedfe9ac6f09/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=src/lib.c
  TARGET = Some("x86_64-unknown-linux-gnu")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = Some("/home/user/works/openwrt/staging_dir/host/bin/gcc")
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = Some("\"-Wl,-rpath-link,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib\"")
  cargo:warning=gcc: warning: "-Wl,-rpath-link,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib": linker input file unused because linking not done
  cargo:warning=gcc: error: "-Wl,-rpath-link,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib": linker input file not found: No such file or directory

  --- stderr


  error occurred: Command "/home/user/works/openwrt/staging_dir/host/bin/gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "\"-Wl,-rpath-link,/home/user/works/openwrt/staging_dir/target-x86_64_glibc_custom/usr/lib/libintl-full/lib\"" "-o" "/home/user/works/openwrt/build_dir/target-x86_64_glibc_custom/einat-ebpf-0.1.3/target/release/build/vsprintf-f6a1be3b8a629995/out/2e40c9e35e9506f4-lib.o" "-c" "src/lib.c" with args gcc did not execute successfully (status code exit status: 1).

@EHfive

This comment was marked as resolved.

@EHfive
Copy link
Owner

EHfive commented Jun 16, 2024

@predators46 Your log and your answer was not helpful. This is not a support forum.

Go read and try follow https://github.com/muink/openwrt-einat-ebpf 's README first. And if that not works even after that, file a separate issue in a proper place with your problem described in detail instead of letting someone else guessing it for you.


@predators46 has deleted their comments, here's the recreated conversation:

predators46: (post only the log suggesting missing libelf, without any description)

EHfive: Why just posting logs? Elaborate it please.

predators46: i am having trouble building in openwrt

EHfive: (this message) Your log and your answer was not helpful...

predators46: ok I missed something

@predators46

This comment was marked as off-topic.

@EHfive
Copy link
Owner

EHfive commented Jun 17, 2024

Closing as this is becoming lengthy and off-topic. And this is not really a issue in the scope of einat development.

It's also spreading the information which would not be reachable by others by default.

Please continue the conversation in issues opened by @vortexilation in OpenWrt's repos. And feel free to tag me there.

@EHfive EHfive closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question This is a question
Projects
None yet
Development

No branches or pull requests

4 participants