uniffi cache not cleared

[mbway]

Bug Description

while working to upgrade the tests of the import hook to the latest maturin ( PyO3/maturin-import-hook#9 ) I ran into an issue where many of the uniffi tests were failing due to missing symbols (failing job). I have been able to narrow down the issue so it can be reproduced without the import hook.

The issue seems to be related to the cache in $CARGO_TARGET_DIR/maturin/uniffi not being cleared between installs resulting in subsequent uniffi packages to be created incorrectly.

the script can be used with USE_MATURIN_NEW_FOO and USE_MATURIN_NEW_BAR set to 0 or 1 to switch between creating new packages and using the provided packages which are based on the maturin/test-crates/uniffi-pure-proc-macro. Both options fail but with different error messages.

set -e
USE_MATURIN_NEW_FOO=0
USE_MATURIN_NEW_BAR=0
CLEAR_CACHE=0

OUT=/tmp/minimal
IN=/tmp/inputs

rm -rf $OUT
python -m venv $OUT/venv
. $OUT/venv/bin/activate
pip install uniffi-bindgen==0.28.0

export CARGO_TARGET_DIR=$OUT/target
export RUST_LOG=maturin=debug

if [ $USE_MATURIN_NEW_FOO -eq 1 ]; then
    maturin new --bindings uniffi --name foo $OUT/foo
else
    cp -r $IN/foo $OUT/foo
fi

cd $OUT/foo
maturin develop
echo "checking foo is installed"
python -c 'import foo; foo.add(1, 2)'

if [ $CLEAR_CACHE -eq 1 ]; then
    echo "clearing cache"
    rm -rf $CARGO_TARGET_DIR
fi

if [ $USE_MATURIN_NEW_BAR -eq 1 ]; then
    maturin new --bindings uniffi --name bar $OUT/bar
else
    cp -r $IN/bar $OUT/bar
fi

cd $OUT/bar
maturin develop
echo "checking bar is installed"
python -c 'import bar; bar.add(1, 2)'

inputs.zip

when cached cleared

when the cache is removed between calls to maturin develop by setting CLEAR_CACHE=1 the script executes successfully.

with provided packages

with USE_MATURIN_NEW_FOO=0 and USE_MATURIN_NEW_BAR=0 the script fails when checking that bar is installed with:

AttributeError: /tmp/minimal/venv/lib/python3.12/site-packages/bar/libexample.so: undefined symbol: uniffi_foo_fn_func_add

(and the word foo does not appear anywhere in the bar package so it shouldn't be looking for the symbol uniffi_foo_fn_func_add)

At the point where maturin develop is run for bar, $CARGO_TARGET_DIR/maturin/uniffi contains foo.py and it is not cleared out when building bar, as can be seen using RUST_LOG=maturin=debug

2024-11-30T21:17:12.698092Z DEBUG build_wheels: maturin::module_writer: Adding bar-0.1.0.dist-info/METADATA
2024-11-30T21:17:12.698801Z DEBUG build_wheels: maturin::module_writer: Adding bar-0.1.0.dist-info/WHEEL
2024-11-30T21:17:12.743188Z DEBUG build_wheels: maturin::module_writer: Running cd "/tmp/minimal/bar" && "uniffi-bindgen" "generate" "--no-format" "--language" "python" "--out-dir" "/tmp/minimal/target/maturin/uniffi" "--library" "/tmp/minimal/target/debug/libbar.so"
2024-11-30T21:17:12.817516Z DEBUG build_wheels: maturin::module_writer: Adding bar/__init__.py
2024-11-30T21:17:12.818344Z DEBUG build_wheels: maturin::module_writer: Adding bar/foo.py from /tmp/minimal/target/maturin/uniffi/foo.py
2024-11-30T21:17:12.819837Z DEBUG build_wheels: maturin::module_writer: Adding bar/bar.py from /tmp/minimal/target/maturin/uniffi/bar.py
2024-11-30T21:17:12.821349Z DEBUG build_wheels: maturin::module_writer: Adding bar/libexample.so from /tmp/target/debug/libbar.so
2024-11-30T21:17:13.051791Z DEBUG build_wheels: maturin::module_writer: Adding bar-0.1.0.dist-info/RECORD
📦 Built wheel to /tmp/.tmpBrngU0/bar-0.1.0-py3-none-linux_x86_64.whl

where foo.py is being added to the bar wheel

with new packages

with USE_MATURIN_NEW_FOO=1 and USE_MATURIN_NEW_BAR=0 the script fails when checking that bar is installed with:

OSError: /tmp/minimal/venv/lib/python3.12/site-packages/bar/libuniffi_example.so: cannot open shared object file: No such file or directory

with USE_MATURIN_NEW_FOO=1 and USE_MATURIN_NEW_BAR=1 the script succeeds because both packages produce an example.py in the cache so the bar version overwrites the foo version and there is no conflict.

Your maturin version (maturin --version)

maturin 1.7.6

Your Python version (python -V)

Python 3.12.2

Your pip version (pip -V)

pip 24.0

What bindings you're using

uniffi

Does cargo build work?

• Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

• Yes

Steps to Reproduce

see above