android qt6 build not reproducible #8746
Milestone
Comments
Looking at libQt6Quick_arm64-v8a.so, it seems that there's mainly offset shifts of 8 or 16 bytes, leading to a large diff but with a single (or at least limited amount of) cause(s). |
|
I tried a build with qt 6.5.3 and ndk 25c (803ec2b), there only 3 files differ. |
interesting. is there a way to build these files individually, without waiting for the entire build? |
|
A bit of a tangent, but also a heads-up, I've wasted lots of hours due to missing this: diff --git a/pythonforandroid/recipes/genericndkbuild/__init__.py b/pythonforandroid/recipes/genericndkbuild/__init__.py
index f114f080ef..f6eff79214 100644
--- a/pythonforandroid/recipes/genericndkbuild/__init__.py
+++ b/pythonforandroid/recipes/genericndkbuild/__init__.py
@@ -10,7 +10,7 @@ class GenericNDKBuildRecipe(BootstrapNDKRecipe):
url = None
depends = ['python3']
- conflicts = ['sdl2', 'qt5']
+ conflicts = ['sdl2', 'qt6']
def should_build(self, arch):
return TrueThis was causing ~non-deterministic build failures for me. Sometimes p4a's dependency graph-resolver (graph.py), to satisfy the "android" recipe, would pick "genericndkbuild" instead of "qt6". To be specific, the "android" recipe specifies the following: So graph.py would try both of these options:
Case 2 should not be valid as qt6 and genericndkbuild should not be picked together -- that results in build issues e.g. for the "pyjnius" recipe (due to e.g. conditional git patches, but AFAICT there are multiple potential ways it can fail based on the order of the other dependencies). graph.py picks case1 or case2 depending on the order the graph is explored, which depends on For debug builds, So that means debug builds would just be failing randomly, independently for each build. Because we are debugging reproducibility issues, I was building release builds, and making temp commits left-and-right. And I was getting the weirdest build failures that looked completely unrelated to what I just changed, and when I reverted to known good code and created a new commit for it, that code too would sometimes fail. I actually considered hardware failure and ran memtest and looked at smartctl... Note that the CI is building debug builds. i.e. there should be lots of random failures for the CI android build task. |
|
(testing with SomberNight@6694ab9, qt6.5.3) The reason for The parent folder contains a dozen cpp files, all start with a large inlined blob followed by some generated functions. However for this one cpp file, in one of two builds, only the inlined binary blob is present, the generated functions are missing: qtquickcontrols2materialstyleimplplugin_RoundedElevationEffect_qml.cpp$ sudo diff /tmp/electrum_build/android{2,3}/fresh_clone/electrum/.buildozer/android/platform/build-armeabi-v7a/dists/Electrum/jni/qt6/qtdeclarative/src/quickcontrols/material/impl/.rcc/qmlcache/qtquickcontrols2materialstyleimplplugin_RoundedElevationEffect_qml.cpp
264a265,280
>
> template <typename Binding>
> void wrapCall(const QQmlPrivate::AOTCompiledContext *aotContext, void *dataPtr, void **argumentsPtr, Binding &&binding)
> {
> using return_type = std::invoke_result_t<Binding, const QQmlPrivate::AOTCompiledContext *, void **>;
> if constexpr (std::is_same_v<return_type, void>) {
> Q_UNUSED(dataPtr)
> binding(aotContext, argumentsPtr);
> } else {
> if (dataPtr) {
> new (dataPtr) return_type(binding(aotContext, argumentsPtr));
> } else {
> binding(aotContext, argumentsPtr);
> }
> }
> }
266c282,349
< extern const QQmlPrivate::TypedFunction aotBuiltFunctions[] = { { 0, QMetaType::fromType<void>(), {}, nullptr } };QT_WARNING_POP
---
> extern const QQmlPrivate::TypedFunction aotBuiltFunctions[] = {
> { 0, QMetaType::fromType<QVariant>(), { },
> [](const QQmlPrivate::AOTCompiledContext *context, void *data, void **argv) {
> wrapCall(context, data, argv, [](const QQmlPrivate::AOTCompiledContext *aotContext, void **argumentsPtr) {
> Q_UNUSED(aotContext)
> Q_UNUSED(argumentsPtr)
> // expression for _shadows at line 10, column 5
> bool r2_1;
> QVariant r2_2;
> int r2_0;
> int r7_0;
> // generate_LoadQmlContextPropertyLookup
> while (!aotContext->loadScopeObjectPropertyLookup(0, &r2_0)) {
> aotContext->setInstructionPointer(2);
> aotContext->initLoadScopeObjectPropertyLookup(0, QMetaType::fromType<int>());
> if (aotContext->engine->hasError())
> return QVariant();
> }
> // generate_StoreReg
> r7_0 = r2_0;
> // generate_GetLookup
> while (!aotContext->getEnumLookup(2, &r2_0)) {
> aotContext->setInstructionPointer(8);
> aotContext->initGetEnumLookup(2, []() { static const auto t = QMetaType::fromName("QQuickMaterialStyle*"); return t; }().metaObject(), "RoundedScale", "NotRounded");
> if (aotContext->engine->hasError())
> return QVariant();
> }
> // generate_CmpStrictEqual
> r2_1 = r7_0 == r2_0;
> // generate_JumpFalse
> if (!r2_1) {
> goto label_0;
> }
> ;
> // generate_LoadQmlContextPropertyLookup
> while (!aotContext->loadScopeObjectPropertyLookup(3, &r2_2)) {
> aotContext->setInstructionPointer(14);
> aotContext->initLoadScopeObjectPropertyLookup(3, QMetaType::fromType<QVariant>());
> if (aotContext->engine->hasError())
> return QVariant();
> }
> // generate_Jump
> {
> goto label_1;
> }
> ;
> label_0:;
> // generate_CallQmlContextPropertyLookup
> {
> QVariant callResult;
> void *args[] = { &callResult };
> const QMetaType types[] = { QMetaType::fromType<QVariant>() };
> while (!aotContext->callQmlContextPropertyLookup(4, args, types, 0)) {
> aotContext->setInstructionPointer(20);
> aotContext->initCallQmlContextPropertyLookup(4);
> if (aotContext->engine->hasError())
> return QVariant();
> }
> r2_2 = std::move(callResult);
> }
> label_1:;
> // generate_Ret
> if (!r2_2.isValid())
> aotContext->setReturnValueUndefined();
> return r2_2;
> });}
> },{ 0, QMetaType::fromType<void>(), {}, nullptr }};
> QT_WARNING_POPdetailed diff (a31 vs a32)(paths within |
Well, I've lost quite some hours on this as well in the past, until I added 'qt5' (and later 'qt6') to the other bootstrap recipes' conflict list. Since then, I've not seen these conflicts arise at all. Since this worked for me, I haven't dug any further into graph.py. Thanks for your analysis, I think the proper fix would be for graph.py to exclude any recipes derived from
I do keep a shell inside the build container open for long stretches of time, and run make_apk.sh directly for a build. Probably, once |
The basic style plugin can probably be excluded from the APK. There's no linker dependency from the Material style we use, unless it is loaded dynamically at runtime, but AFAIR this is not done. |
|
When generating compiled resources using Details``` /home/user/wspace/electrum/.buildozer/android/platform/build-arm64-v8a/build/other_builds/hostqt6/native-build/install/libexec/qmlcachegen --bare --resource-path /qt-project.org/imports/QtQuick/Controls/Material/Button.qml -I /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtbase/./qml -I /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtbase/./qml -i /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtbase/qml/QtQuick/Controls/Material/qmldir --resource /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtdeclarative/src/quickcontrols2/material/.rcc/qmake_QtQuick_Controls_Material.qrc --resource /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtdeclarative/src/quickcontrols2/material/.rcc/qtquickcontrols2materialstyleplugin_raw_qml_0.qrc --resource /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtdeclarative/src/quickcontrols2/material/.rcc/qtquickcontrols2materialstyleplugin.qrc -o /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtdeclarative/src/quickcontrols2/material/.rcc/qmlcache/qtquickcontrols2materialstyleplugin_Button_qml.cpp /mnt/buildozer/.buildozer/android/platform/build-arm64-v8a/build/bootstrap_builds/qt6/jni/qt6/qtdeclarative/src/quickcontrols2/material/Button.qml ```
.. and Qt claims to support the However, on each successive run, the output is different. (this is with Qt6.4.3) Edit: the hash seed hack used in the code for |
|
The cache generator is not generating code deterministicly, but I think I have a patch for qmlcachegen to produce deterministic output. I'll do a full compile first to see if nothing fell over. |
|
patch here: accumulator/python-for-android@a6e4f1b |
|
Nice! Great find! detailed diff (a11 vs a12)(paths within |
|
(testing with 50d3c52) With qt6.4.3, applying the patch for this bug from qt6.5: The diff is now: EDIT: detailed diff (a21 vs a22)(paths within When building with qt6.5.3, all qsb files are deterministic (none appear in diff in #8746 (comment)), so hopefully there is some other patch (and not a large refactor) that we could just "backport". |
SomberNight
added a commit
to SomberNight/electrum
that referenced
this issue
Dec 16, 2023
This is that patch, extracted from a large changeset merged into qt6.5.0: With that, I can now reproducibly build the full armeabi-v7a apk, with qt6.4.3. EDIT: I only tested one arch, but hopefully master can now be built reproducibly, at bf4934b.
I wonder if we could just use |
|
thank both of you for fixing this so quickly |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Building Android from the 4.5.0b0 tag (f4cddd7), using
the apk is not reproducible.
The following files differ:
The differences look non-trivial and not localised to small sections.
related #8545
The text was updated successfully, but these errors were encountered: