Failed to build mbedtls shared library

[Ra2-IFV]

Summary

Failed to build mbedtls shared library.

System information

Mbed TLS version (number or commit id): 3.6.0
Operating system and version: Debian 12.2 in VirtualBox
Configuration (if not default, please attach mbedtls_config.h): full with config.py
Compiler and options (if you used a pre-built binary, please indicate how you obtained it): gcc version 12.2.0 (Debian 12.2.0-14)
Additional environment information:

Command

cmake -DCMAKE_INSTALL_PREFIX=$(pwd)/build -DENABLE_TESTING=OFF -DENABLE_PROGRAMS=ON -DUSE_SHARED_MBEDTLS_LIBRARY=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON .
make

Log

......
[ 92%] Building C object programs/test/CMakeFiles/dlopen.dir/dlopen.c.o
[ 92%] Linking C executable dlopen
/usr/bin/ld: CMakeFiles/dlopen.dir/dlopen.c.o: warning: relocation against `mbedtls_printf' in read-only section `.text'
/usr/bin/ld: CMakeFiles/dlopen.dir/dlopen.c.o: in function `main':
dlopen.c:(.text+0x69): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0xd4): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x111): undefined reference to `mbedtls_printf'
/usr/bin/ld: dlopen.c:(.text+0x187): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x1f0): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x25b): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x269): undefined reference to `mbedtls_printf'
/usr/bin/ld: dlopen.c:(.text+0x2e2): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x34b): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x3b6): undefined reference to `mbedtls_exit'
/usr/bin/ld: dlopen.c:(.text+0x3f9): undefined reference to `mbedtls_printf'
/usr/bin/ld: dlopen.c:(.text+0x478): undefined reference to `mbedtls_exit'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
make[2]: *** [programs/test/CMakeFiles/dlopen.dir/build.make:97: programs/test/dlopen] Error 1
make[1]: *** [CMakeFiles/Makefile2:2491: programs/test/CMakeFiles/dlopen.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

[mpg]

Hi @Ra2-IFV and thanks for your report. With the details you provided I was able to reproduce the build failure.

Another way to reproduce (which I'd say points to a gap in our testing) is to add scripts/config.py full in component_test_cmake_shared in all.sh - or just scripts/config.py set MBEDTLS_PLATFORM_EXIT_ALT is enough (giving only a subset of the error messages of course).

From a cursory glance I'd say use of mbedtls_exit and mbedtls_printf in this specific test program is questionable in light of this kind of configuration. @gilles-peskine-arm since you wrote this program, what do you think?

[mpg]

Note: the PR that introduced this program was merged in 3.1.0 so I don't think this is new in 3.6 (to be confirmed).

[gilles-peskine-arm]

About dlopen and platform functions

I'd say use of mbedtls_exit and mbedtls_printf in this specific test program is questionable in light of this kind of configuration.

It's true that any platform that has dlopen() surely doesn't need custom exit and printf functions. However, even if we don't use the platform functions directly in dlopen.c, they're used in the library, so running the dlopen program will fail. So I think it's actually a good thing that they're used.

The full test — which we aren't doing because #2698 is unfinished — is to check that we can compile and run a program that uses the shared library. make SHARED=1 && programs/test/dlopen_demo.sh or minimally:

make SHARED=1 lib &&
make SHARED=1 -C programs test/dlopen &&
LD_LIBRARY_PATH=$PWD/library${LD_LIBRARY_PATH:+:}${LD_LIBRARY_PATH} programs/test/dlopen

This works in the default configuration, but not in configurations that rely on custom implementations on platform functions. Platform functions can be provided in a variety of ways:

• If they're part of the standard library, all just works.
• If they're in some other dynamically loadable library, dlopen.c would need to load that library first. (Actually those libraries: there could be more than one.)
• If they're supposed to be linked statically into the application program, then we need to provide the right arguments when linking dlopen. For user-provided objects, that will be in LDFLAGS. Objects that we provide ourselves are a special case because it's our job to set up LDFLAGS correctly.

config.py full enables platform functions that are implemented in tests/src/**.c and that we normally link statically. But we're not equipped to do that, because tests/src/**.c have dependencies on libmbedcrypto. We would need to distinguish the test objects that implement platform functions and don't require libmbedcrypto functions from the test objects that implement test helper functions and might require libmbedcrypto functions.

In summary, this is a bug in our build scripts (both make and cmake), which is hard to resolve.

About the full config

config.py full is not really a user-facing interface: it's intended for testing. If you want to enable experimental features but not platform interfaces, use config.py full_no_platform.

I'm inclined to consider this a low-priority defect in a test script, and to classify it as very low priority due to the high difficulty and low impact.

[mpg]

I'm inclined to consider this a low-priority defect in a test script, and to classify it as very low priority due to the high difficulty and low impact.

Seems fair to me. I'm almost tempted to consider it "won't fix" - but then document it. Like, dlopen.c could check if MBEDTLS_PLATFORM_EXIT_ALT or MBEDTLS_PLATFORM_PRINTF_ALT is set, and compile a dummy main() that just prints a message if they are.

Also, I think in component_test_cmake_shared we might want to use config.py full_no_platform.

config.py full is not really a user-facing interface

@Ra2-IFV out of curiosity, how did you come to use config.py full and what was your goal in doing so? Would you be happy just using full_no_platform or something else instead?

[Ra2-IFV]

Hi, thanks for your kindness and quick response!
Actually, I was having problems with curl 8.8.0 and mbedtls 3.6.0 in OpenWRT, see openwrt/packages#24386 openwrt/packages#24365 , so I tried to build them manually to see if it can be reproduced on other platforms.
I selected full option because missing components may lead to other errors, which makes it harder to diagnose. I only took a few minutes to read the docs so I don't know much about mbedtls, and what these presets are used for. Apologies.

[Ra2-IFV]

Tried all options and only full_no_platform passed.

[mpg]

I only took a few minutes to read the docs so I don't know much about mbedtls, and what these presets are used for. Apologies.

You have nothing to apologise for: if you had taken more time to read more docs I'm not sure you'd have found anything about this anyway - in fact I'm preparing a small PR about that.

Regarding the original issue, if you have TLS connections that were working before 3.6.0 and are now failing, I'd recommend having a look at #9223 - unfortunately we missed quite a few backwards compatibility concerns when enabling TLS 1.3 by default in 3.6 and that has been causing issues for several users. Hopefully this should be all fixed in 3.6.1 (no firm date yet, but should be next quarter).

[Ra2-IFV]

Thanks, I already read that. At least there is a workaround in curl and works as expected, so I can still enjoy the benefits of mbedtls.
Thanks for contributing this project :)