Build fails for native_posix board when using C++ <atomic> header

[palchak-google]

Describe the bug
Build generates a hard error when targeting the native_posix or native_posix_64 board if the code includes the C++ standard library header <atomic>.

To Reproduce
Steps to reproduce the behavior:

1. Create an main.cc file as follows:

#include <atomic>
void main() {}

2. Set the following options in prj.conf:

CONFIG_CPLUSPLUS=y
CONFIG_LIB_CPLUSPLUS=y
CONFIG_STD_CPP20=y

3. Build: west build --board=native_posix
4. Observe the following build error:

In file included from /usr/include/c++/11/bits/atomic_base.h:41,
                 from /usr/include/c++/11/atomic:41,
                 from /usr/local/google/home/palchak/projects/zephyr-upstream/app/gcc11/main.cc:1:
/usr/include/c++/11/bits/atomic_wait.h: In function ‘void std::__detail::__platform_wait(const _Tp*, std::__detail::__platform_wait_t)’:
/usr/include/c++/11/bits/atomic_wait.h:104:29: error: ‘SYS_futex’ was not declared in this scope
  104 |         auto __e = syscall (SYS_futex, static_cast<const void*>(__addr),
      |                             ^~~~~~~~~
/usr/include/c++/11/bits/atomic_wait.h: In function ‘void std::__detail::__platform_notify(const _Tp*, bool)’:
/usr/include/c++/11/bits/atomic_wait.h:117:18: error: ‘SYS_futex’ was not declared in this scope
  117 |         syscall (SYS_futex, static_cast<const void*>(__addr),
      |                  ^~~~~~~~~
[91/99] Building C object zephyr/kernel/CMakeFiles/kernel.dir/sched.c.obj
ninja: build stopped: subcommand failed.

Expected behavior
Build should be complete without error.

Impact
Unable to target native_posix and native_posix_64 boards for testing and development.

Environment (please complete the following information):

• OS: Linux
• Toolchain: GCC 11.2.0-8
• Zephyr v2.7.0-rc3

Additional context
This issue does not occur with GCC 10.3.0. It is the result of libstdc++ adding support for std::atomic<T>::wait. Atomic wait functionality was introduced with the C++20 standard. When built for Linux libstdc++ uses the kernel's futex API to perform the waiting.

[cfriedt]

@stephanosio - is this a bug or a feature we simply haven't added yet?

[stephanosio]

@stephanosio - is this a bug or a feature we simply haven't added yet?

@cfriedt This happens because the Linux libstdc++ included in the host GCC 11 now directly makes use of the Linux futex for the atomics.

We will need to see if there is a way to globally disable the use of Linux futex, or implement an abstraction for it in the native_posix "SoC" somehow.

p.s. as for whether this is a bug or an enhancement, it is hard to say because we do not currently have a supported C++ standard-features-compiler type and version matrix; but, I am inclined to say it is a bug since it does break something that used to work.

[palchak-google]

Just yesterday I was successful in implementing a local work-around for this issue. The direct dependency within libstdc++ on the Linux futex API is conditionally enabled by the preprocessor symbol _GLIBCXX_HAVE_LINUX_FUTEX. If this symbol is not defined, libstdc++ will fall back to using an ordinary mutex.

The symbol _GLIBCXX_HAVE_LINUX_FUTEX is defined in a file named c++config.h, and there are versions of this file for different machine targets (e.g. -m32, -mx32, etc). On my system the file for -m32 targets is located at /usr/include/x68_64-linux-gnu/c++/11/32/bits/c++config.h. I manually edited the file on my system to add a conditional check as follows:

/* Define if futex syscall is available. */
#ifndef __ZEPHYR__
#define _GLIBCXX_HAVE_LINUX_FUTEX 1
#endif

Of course, this isn't feasible as a final solution, but it's a start. What should be possible is for Zephyr to provide a c++config.h file that does something like this:

#include_next <bits/c++config.h>
#ifdef _GLIBCXX_HAVE_LINUX_FUTEX
#undef _GLIBCXX_HAVE_LINUX_FUTEX
#endif

Of course, the path to Zephyr's c++config.h would have to be prepended to the list of include paths so that Zephyr's c++config.h is found first by the other library headers. I haven't tried to do this yet, so I don't know if it will actually work.

[palchak-google]

So I tried creating a parallel c++config.h file, and that appeared to work perfectly. I added the file zephyr/boards/posix/native_posix/bits/c++config.h with the following contents:

#include_next <bits/c++config.h>

#if defined(__ZEPHYR__) && defined(_GLIBCXX_HAVE_LINUX_FUTEX)
#undef _GLIBCXX_HAVE_LINUX_FUTEX
#endif

I reverted my local customization to the system-wide c++config.h file, and I was still able to build for the native_posix target without error.

This seems like a viable path for resolving this issue.

Beyond fixing the futex issue, there still might be some additional things that could be added to Zephyr related to this topic. In the atomic_wait.h header for libstdc++, there is the following comment:

#ifdef _GLIBCXX_HAVE_LINUX_FUTEX
...
#else
// define _GLIBCX_HAVE_PLATFORM_WAIT and implement __platform_wait()
// and __platform_notify() if there is a more efficient primitive supported
// by the platform (e.g. __ulock_wait()/__ulock_wake()) which is better than
// a mutex/condvar based wait
#endif

While falling back on a mutex when building for the native_posix target is perfectly acceptable, Zephyr should provide a Zephyr-specific implementation of __platform_wait() and __platform_notify() instead. The reason I say should is that the GNU ARM Embedded (aka gcc-arm-none-eabi) toolchain also ships with libstdc++. Thus ARM platforms (and any others that use libstdc++) would also and benefit from Zephyr-specific implementations of these platform functions. This of course would be a feature enhancement and deserves its own tracking issue, but as it's related to the Linux futex bug it warranted a mention here.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[palchak-google]

Not stale. Still an issue.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[RafaelLaya]

Bump this was closed without resolution