--sdk no longer works in latest Ubuntu PPA

[davidchisnall]

Xmake Version

v2.9.6+20241030

Operating System Version and Architecture

Ubuntu (x86-64 and AArch64)

Describe Bug

Out dev container build does a smoke test to make sure that it can build the RTOS. The builds from yesterday bring a new version of xmake from the Ubuntu PPA and now we see compile failures.

It appears that the latest xmake no longer finds our clang. This was working with versions of our dev container built three days ago. Those included xmake v2.9.5+20240914.

Expected Behavior

We run xmake f --sdk=/cheriot-tools/, where /cheriot-tools/bin contains the toolchain files (clang, lld, and so on). This should then compile with the clang that we have found there.

Project Configuration

https://github.com/CHERIoT-Platform/cheriot-rtos/blob/main/tests/xmake.lua

Most of the rules come from here, including the toolchain definition:

https://github.com/CHERIoT-Platform/cheriot-rtos/blob/main/sdk/xmake.lua

The dev container image with the toolchain installed is available here:

https://github.com/orgs/CHERIoT-Platform/packages/container/package/devcontainer

The smoke-test build is done as:

cd /tmp
git clone --recurse http://github.com/Microsoft/cheriot-rtos
xmake f --sdk=/cheriot-tools/
xmake
xmake run

This works, but if you install a new xmake it does not (unfortunately, changes to the Ubuntu base layer broke sudo - I can fix that once I have a working xmake package).

Additional Information and Error Logs

$ xmake f --sdk=/cheriot-tools/ -v
checking for platform ... cheriot
checking for architecture ... cheriot
generating /tmp/cheriot-rtos/sdk/firmware.ldscript.in to build/test-suite-firmware.ldscript ..
checking for git ... /usr/bin/git
generating /tmp/cheriot-rtos/sdk/firmware.ldscript.in ... ok
configure
{
    debug-locks = false
    board = sail
    host = linux
    mode = release
    stack-usage-check-allocator = false
    debug-loader = false
    debug-allocator = false
    ndk_stdcxx = true
    stack-usage-check-scheduler = false
    arch = cheriot
    debug-scheduler = false
    kind = static
    debug-cxxrt = false
    sdk = /cheriot-tools/
    target_os = linux
    plat = cheriot
    scheduler-accounting = false
    debug-token_library = false
    buildir = build
    ccache = true
}
$ xmake -v
checking for clang++ ... no
checking for the c++ compiler (cxx: clang++) ... no
checking for clang ... no
checking for the assember (as: clang) ... no
checking for clang ... no
checking for the c compiler (cc: clang) ... no
checking for gcc ... /usr/bin/gcc
checking for the c++ compiler (cxx) ... gcc
checking for gcc ... /usr/bin/gcc
checking for the assember (as) ... gcc
checking for gcc ... /usr/bin/gcc
checking for the c compiler (cc) ... gcc
checking for /usr/bin/gcc ... ok
checking for flags (-fPIC) ... no
[ 29%]: cache compiling.release allocator-test.cc
/usr/bin/gcc -c -std=c++20 -target riscv32-unknown-unknown -mcpu=cheriot -mabi=cheriot -mxcheri-rvc -mrelax -fshort-wchar -nostdinc -Oz -g -ffunction-sections -fdata-sections -fomit-frame-pointer -fno-builtin -fno-exceptions -fno-asynchronous-unwind-tables -fno-c++-static-destructors -fno-rtti -Werror -I/tmp/cheriot-rtos/sdk/include/c++-config -I/tmp/cheriot-rtos/sdk/include/libc++ -I/tmp/cheriot-rtos/sdk/include -I/tmp/cheriot-rtos/sdk/boards/../include/platform/generic-riscv -DTEMPORAL_SAFETY -DSOFTWARE_REVOKER -DSAIL -DCPU_TIMER_HZ=2000 -DTICK_RATE_HZ=10 -DSIMULATION -DCONFIG_MSHWM -DDEVICE_EXISTS_clint -DDEVICE_EXISTS_shadow -DDEVICE_EXISTS_uart -DREVOKABLE_MEMORY_START=0x80000000 "-DCHERIOT_INTERRUPT_NAMES=FakeInterrupt=4, " -cheri-compartment=allocator_test -o build/.objs/allocator_test/cheriot/cheriot/release/allocator-test.cc.o allocator-test.cc

For comparison, with v2.9.5+20240914, we see this:

$ xmake f --sdk=/cheriot-tools/ -v
checking for platform ... cheriot
checking for architecture ... cheriot
generating /tmp/cheriot-rtos/sdk/firmware.ldscript.in to build/test-suite-firmware.ldscript ..
checking for git ... /usr/bin/git
generating /tmp/cheriot-rtos/sdk/firmware.ldscript.in ... ok
configure
{
    board = sail
    debug-locks = false
    debug-cxxrt = false
    kind = static
    stack-usage-check-scheduler = false
    arch = cheriot
    scheduler-accounting = false
    mode = release
    ndk_stdcxx = true
    sdk = /cheriot-tools/
    debug-loader = false
    ccache = true
    target_os = linux
    debug-token_library = false
    debug-allocator = false
    stack-usage-check-allocator = false
    debug-scheduler = false
    plat = cheriot
    buildir = build
    host = linux
}
$ xmake -v
checking for clang++ ... /cheriot-tools/bin/clang++
checking for the c++ compiler (cxx) ... clang++
checking for clang ... /cheriot-tools/bin/clang
checking for the c compiler (cc) ... clang

And then the build works, because it's using the correct compiler.

[nwf]

Bisecting in the devcontainer suggests that 7af1ca5 is the first bad commit.

Automated reproducer

For the curious or perhaps future me...

Setup

cd /tmp
git clone --recurse https://github.com/xmake-io/xmake/
git clone --recurse http://github.com/Microsoft/cheriot-rtos
(cd cheriot-rtos; git reset --hard 6f3b7d9; git submodule update)

cat >/tmp/build.sh <<HERE
#!/bin/sh

set -e -u

rm -rf /tmp/xmake.inst
rm -rf /tmp/tmp.*
rm -rf /tmp/.xmake*

cd /tmp/xmake
git clean -xfd
git submodule update
./configure --prefix=/tmp/xmake.inst
make -j4 build
make install

export PATH=/tmp/xmake.inst/bin:$PATH
cd /tmp/cheriot-rtos/tests
git clean -xfd
xmake f --sdk=/cheriot-tools
xmake || exit 1
HERE
chmod +x /tmp/build.sh

Run

cd /tmp/xmake
git bisect start
git bisect good v2.9.5
git bisect bad v2.9.6
git bisect run /tmp/build.sh

[waruqi]

try this patch. #5780

xmake update -s dev

[nwf]

Yes, testing with eada026, which includes the merge of #5780 (1c689fd), works. Thank you!

[davidchisnall]

Would it be possible to include this fix in the PPA package? We can't update our dev container currently.

[waruqi]

Would it be possible to include this fix in the PPA package? We can't update our dev container currently.

I don't know how to update it without updating the version?

you can run

xmake update dev
source ~/.xmake/profile

to update to dev on container.

[davidchisnall]

I don't know how to update it without updating the version?

Maybe do a point release? Or bump the date suffix? Or keep the old version available so that we can pin the version and have one that can still build any of our things?

to update to dev on container.

This isn't easy to integrate with a dev container.

[waruqi]

try 2.9.6+2

[davidchisnall]

That works, thanks! I’ll try to set up a nightly CI job to test xmake head and catch things like this prior to release in the future.

[davidchisnall]

FYI: I've set up a nightly job for us to test with the latest xmake:

https://github.com/CHERIoT-Platform/cheriot-rtos/actions/workflows/test-new-xmake-nightly.yml

It's currently failing because the sudo config is broken in our dev container, but that will be fixed as soon as the dev container is rebuilt (20-30 minutes).