Unable to depend on any crate using cc or bindgen (std only)

[jasta]

Bug description

After completing the espup setup steps here: https://esp-rs.github.io/book/installation/installation.html, a user is unable to depend on any crate that uses cc or bindgen (i.e. any crate that uses C code). This also includes using your own C code in your project with a simple build.rs utilizing the cc crate.

There are several distinct problems here, and I'll attempt to outline them each with tracking issues:

• export-esp.sh does not set CLANG_PATH which can confuse bindgen (and any crate using clang-sys) #164
• How to provide TARGET_CC/CXX/AR when cross compiling? #165
• cc-rs #776: cc missing *-espidf compiler path prefixes
• rust-bindgen #2396: bindgen lacks a mapping from the rust triple {riscv32imc,xtensa}-esp*-espidf to CLang triple (e.g. riscv32imc-esp-espidf to riscv32-esp-elf)

Would you like to work on a fix? Yes

To Reproduce

Steps to reproduce the behavior:

1. Follow steps in Add CC and AR variables of compiler name to generated export script rust-build#61 (comment) to repro all issues other than bindgen
2. Follow https://liufuyang.github.io/2020/02/02/call-c-in-rust.html to reproduce the bindgen issue.

Expected behavior

It's expected that esp-rs projects work when adding dependencies to other crates which utilize C/C++ code via the standard tool paths (cc and bindgen at a minimum).

Screenshots

N/A

Environment

• OS: Ubuntu 22.04
• espup version: 0.2.5

Additional context

Note: I've fixed most of the issues already but need some time to organize the PRs and tracking issues.

[SergioGasquez]

Hello!

Thanks for documenting and filling in the issues! Just a few notes:

• I'd say that this only applies to std projects
• GCC toolchains, which install the binaries required for TARGET_CC, TARGET_CXX and TARGET_AR can also be installed by esp-idf-sys
  • So, fixing this issues here might not be sufficient.
  • For no_std, espup also installs the GCC toolchains as it's used as linker, but this will be changed once LLD is supported
• espup next steps are being decided. If we end up going for the "minimal" approach, I'm not sure if we should try to fix this here. If in the end, we go for the "extended" version this is something we should probably fix.

But, in any case, I think it would be very beneficial to have some documentation in the book

[jasta]

I have only been testing the std use case so that certainly makes sense. All of the problems I ran into have reasonable local solutions available (except the bindgen patch I needed), so once that's landed I think the problem can be neatly wrapped up with some minor fixes to espup (i.e. add export CLANG_PATH=...) and document the need for setting TARGET_* or CROSS_COMPILE (turns out CROSS_COMPILE=riscv32-esp-elf will do that all for you). It may even be possible to patch cc-rs to understand the *-espidf targets and try to find a toolchain prefix automatically. It is seemingly doing this for lots of other targets. Let me explore that further.

[jasta]

Turns out the cc-rs patch I added (rust-lang/cc-rs#776) neatly wraps up the complete solution for riscv32-esp-espidf at least. So with cc-rs, rust-bindgen, and espup patched, users could just invoke cargo build and everything works correctly with dependencies utilizing cc or bindgen (assuming rust-toolchain.toml and .cargo/config.toml are set up right).

[jasta]

@SergioGasquez reading the discussion more closely, I wonder if perhaps there's another more elegant way to fix #164. There's a few separate issues here, so let me try to be specific:

Regarding only the use of clang for bindgen, the issue I'm running into without espup's clang is that my host clang gives a bunch of errors like:

  /home/jasta/.cargo/git/checkouts/lvgl-rs-4808a7bf651c71c2/2ff094e/lvgl-sys/vendor/lvgl/src/lv_misc/lv_area.h:17:10: fatal error: 'string.h' file not found
  thread 'main' panicked at 'Unable to generate bindings: ClangDiagnostic("/home/jasta/.cargo/git/checkouts/lvgl-rs-4808a7bf651c71c2/2ff094e/lvgl-sys/vendor/lvgl/src/lv_misc/lv_area.h:17:10: fatal error: 'string.h' file not found\n")', /home/jasta/.cargo/git/checkouts/lvgl-rs-4808a7bf651c71c2/2ff094e/lvgl-sys/build.rs:128:10

By simply setting CLANG_PATH to the clang provided by espup the issue goes away? I don't even need LIBCLANG_PATH. Maybe this is just a PEBKAC issue, but it's hard to see how right now given all the magic in Rust's cross compilation story :)

Regarding only the gcc toolchain for esp32, you're correct that these tools are provided in the .embuild directory already so my project can simply have a script that adds, e.g. ./.embuild/espressif/tools/riscv32-esp-elf/esp-2021r2-patch5-8.4.0/riscv32-esp-elf/bin/ to PATH. That issue can be safely ignored then.

[SergioGasquez]

I don't even need LIBCLANG_PATH.

Do you any other clang installed in your system other than espressif clang? If I'm not mistaken, LIBCLANG_PATH is only required in this scenario.

I will look to your awesome research when I get back from vacations. Thanks a lot for all your efforts!

[jasta]

Do you any other clang installed in your system other than espressif clang? If I'm not mistaken, LIBCLANG_PATH is only required in this scenario.

Yes, I have clang-14 from Ubuntu. Digging deeper what's happening here is that even though my host clang supports riscv32 as a target, it doesn't have all the includes and such to realistically compile anything. The expectation is that you can provide some more functional gcc toolchain with all this stuff and then point clang at that to do real work. The espressif packaging of clang does this all for you and bakes the configuration into the binary so it "just works" for both host and target compilation (of xtensa or riscv32).

So espup (or something like it) it seems is important even in the world of full Rust and clang support for these targets because you'll still want a way to provide users with a functional clang toolchain for cross compilation. In an ideal world I'd expect rustup to handle this, however. That is, if I tell rustup to install a toolchain for riscv32imc-unknown-none-elf, I expect to be able to compile both pure Rust crates and sys crates utilizing C code targeting that platform. Right now I only get the former.

I will look to your awesome research when I get back from vacations. Thanks a lot for all your efforts!

Just hoping to remove some of the barriers for others getting into this space. I'm just starting out with MCU programming and was hoping Rust could shave off some of the rough edges. For the most part it definitely has, but it added a few sharp corners too :)

[jasta]

All wrapped up for RISC-V targets. Xtensa support needs a change in cc-rs once it's made tier 3. To work around all you need is export CROSS_COMPILE=xstensa-esp32-elf.

[SergioGasquez]

Thanks for all your research and improving espup! We will revisit this issue once Xtensa changes are upstreamed