Integrate Android integration tests into CI pipeline #230

nilsreichardt · 2022-05-17T12:15:07Z

Closes #176

github-actions · 2022-05-17T12:20:48Z

Visit the preview URL for this PR (updated for commit 73419a2):

https://sharezone-test--pr230-add-android-integrat-20ba6kx5.web.app

_{(expires Sat, 08 Oct 2022 23:13:54 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

Jonas-Sander

See above.

Maybe also add a comment that we might chache the AVD image to make it start up faster? (See emulator-runner docs)?

.github/workflows/main.yml

Jonas-Sander · 2022-05-17T15:36:18Z

.github/workflows/main.yml

+      - name: Run integration tests
+        uses: reactivecircus/android-emulator-runner@v2
+        env:
+          USER_1_EMAIL: ${{ secrets.INTEGRATION_TEST_USER_1_EMAIL }}
+          USER_1_PASSWORD: ${{ secrets.INTEGRATION_TEST_USER_1_PASSWORD }}
+        with:
+          api-level: ${{ matrix.api-level }}
+          target: ${{ matrix.target }}
+          arch: ${{ matrix.arch }}
+          profile: pixel_3a
+          emulator-options: -no-snapshot -no-window -no-boot-anim -camera-back virtualscene -camera-front emulated
+          working-directory: ./app
+          # We can not use a multiline command because the
+          # "android-emulator-runner" doesn't support it. It just takes every
+          # line as separate command.
+          script: flutter test integration_test --flavor dev --dart-define USER_1_EMAIL=$USER_1_EMAIL --dart-define USER_1_PASSWORD=$USER_1_PASSWORD


Kinda feel like we should put this into our Sharezone CLI in the future - this would also help to run integration tests locally.
We might even automatically retrieve the secrets inside the script (or at least make it nicer by reading from an .env or sth)

Yes, we can integrate this later in the CLI 👍 But I would like to do a rework of the CLI first and do things like better styling when printing to the console and writing much more tests.

## Description For now, it was required to have a `key.properties` file with our code signing credentials when you tried to build for Android (even when running in debug mode where no code signing was done). We now check if a `key.properties` is present. If no, don't try to code sign the android app, which is the expected behavior because you need code signing only for publishing. This PR unblocks #230 Closes #234

## Description This should fix #237. I ran a Codemagic build and this worked 👍 This unblocks: * #230 * #235

…SharezoneApp/sharezone-app into add-android-integration-test-ci

nilsreichardt · 2022-05-18T15:02:02Z

Tests are failing because of:


══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following MissingPluginException was thrown running a test:
MissingPluginException(No implementation found for method Messaging#requestPermission on channel
plugins.flutter.io/firebase_messaging)

When the exception was thrown, this was the stack:
#0      convertPlatformException (package:firebase_messaging_platform_interface/src/method_channel/utils/exception.dart:13:5)
#1      MethodChannelFirebaseMessaging.requestPermission (package:firebase_messaging_platform_interface/src/method_channel/method_channel_messaging.dart:286:13)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)

The test description was:
  User should be able to sign in
════════════════════════════════════════════════════════════════════════════════════════════════════

Hmm, it seems so that we need to upgrade Firebase Messaging and Firebase Core to unblock this PR: https://stackoverflow.com/questions/69880191/missingpluginexceptionno-implementation-found-for-method-messagingrequestpermi

…on-test-ci

This reverts commit bff3de4.

…on-test-ci

Jonas-Sander · 2022-09-30T18:42:42Z

Just want to say that some android actions fail but because of an firebase auth error, e.g. https://github.com/SharezoneApp/sharezone-app/actions/runs/3160777863/jobs/5145617181

github-actions · 2023-02-22T18:06:56Z

Visit the preview URL for this PR (updated for commit 63d1030):

https://sharezone-test--pr230-add-android-integrat-or4aeld9.web.app

_{(expires Fri, 03 Mar 2023 10:03:55 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

_{Sign: 4cb3ae61e1e018abfd9841fd3239f5b49ccc034b}

nilsreichardt · 2023-02-23T14:38:31Z

Using flutter test instead of flutter drive seams to fix the problem with the Android Emulator. Instead of 30% - 40% failure rate we now have only ~6%. I'm testing the google_atd. When there is everything fine, we can merge this :)

Sharezone-Admin · 2023-02-23T23:19:23Z

⬇️ Generated builds by Codemagic for commit 63d1030 ⬇️

Note: Only Sharezone team members are able to install the iOS app.

iOS	Android
Download link	Download link

Jonas-Sander

See nits above.

Jonas-Sander · 2023-02-23T23:40:33Z

.github/workflows/main.yml

+          # We use the `flutter drive` instead of the `flutter test` command
+          # because the test command times out after 12 minutes. But building the
+          # app takes more than 12 minutes... It seems so that there is no way to
+          # set the timeout (the --timeout argument has not effect). Tracking
+          # issue: https://github.com/flutter/flutter/issues/105913
+          script: flutter test integration_test --flavor dev --dart-define USER_1_EMAIL=$USER_1_EMAIL --dart-define USER_1_PASSWORD=$USER_1_PASSWORD


The comment is out-of-date (you're using flutter test here, not flutter drive)

Jonas-Sander · 2023-02-23T23:41:49Z

.github/workflows/main.yml

+          # We use some emulator options for the following reasons:
+          # * "-no-boot-anim": Disables the boot animation for faster booting.
+          # * "-no-window": Disables the graphical windows, so no display is
+          #   required.
+          # * "-no-snapshot": Disables the quick boot feature. Therefore, the
+          #   emulator does not load or save the emulator state. We want to have
+          #   a fresh testing environment for every test run.
+          # * "-camera-back virtualscene": Shows this virtual room when opening
+          #   the back camera where you can walk. Default option for emulators
+          #   created by Android Studio. We use this so we have the same
+          #   environment locally as when using this action. Is needed when
+          #   testing things with the camera. It's also possible to inject
+          #   images into the virtual scene to test things like qr code scanner.
+          # * "-camera-front": Shows this green monster when opening the front
+          #   camera. Default option for emulators created by Android Studio. We
+          #   use this so we have the same environment locally as when using
+          #   this action. Is needed when testing things with camera.
+          emulator-options: -no-snapshot -no-window -no-boot-anim -camera-back virtualscene -camera-front emulated


Teeny tiny nit: Could you use the same order for the emulator-options: flags and for the comment above explaining them?

…SharezoneApp/sharezone-app into add-android-integration-test-ci

Integrate Android integration tests into CI pipeline

7c3cfb6

nilsreichardt requested a review from Jonas-Sander as a code owner May 17, 2022 12:15

Put test command into one line

d116dcf

Jonas-Sander reviewed May 17, 2022

View reviewed changes

nilsreichardt mentioned this pull request May 17, 2022

Don't require key.properties anymore to build for Android #236

Merged

Merge branch 'main' into add-android-integration-test-ci

3a6a812

github-actions bot added the ci/cd label May 17, 2022

This was referenced May 17, 2022

Android builds are failing in CI environments #237

Closed

Bump fast_rsa to 3.4.7 #239

Merged

nilsreichardt added a commit that referenced this pull request May 17, 2022

Bump fast_rsa to 3.4.7 (#239)

e8bf9db

## Description This should fix #237. I ran a Codemagic build and this worked 👍 This unblocks: * #230 * #235

nilsreichardt added 3 commits May 17, 2022 22:11

Merge branch 'main' into add-android-integration-test-ci

57a30a6

Use subosito/flutter-action@main

7e0a4fc

Merge branch 'add-android-integration-test-ci' of https://github.com/…

047a315

…SharezoneApp/sharezone-app into add-android-integration-test-ci

nilsreichardt mentioned this pull request May 18, 2022

Fix Android builds for CI machines & drop for support macOS video player #241

Merged

nilsreichardt added 2 commits May 18, 2022 16:11

Merge branch 'main' into add-android-integration-test-ci

af8c261

Add better comment

ed61912

nilsreichardt added 8 commits June 13, 2022 18:42

Try google_atd

7b85000

Merge remote-tracking branch 'origin/main' into add-android-integrati…

ebafab1

…on-test-ci

Use better configs for google_atd

8978f15

disable request

9be9acb

Merge branch 'main' into add-android-integration-test-ci

948d58f

Skip requestPermission for Android when running integration test

f540213

Remove matrix

bff3de4

Revert "Remove matrix"

fd034fc

This reverts commit bff3de4.

nilsreichardt marked this pull request as draft June 14, 2022 10:39

nilsreichardt added 2 commits June 14, 2022 12:41

Remove matrix again

c329337

Add comment

169f0b0

nilsreichardt and others added 6 commits June 16, 2022 21:40

Test flaky

f21e40b

Enable for draft PR

45231c8

Using flutter drive command

20f7a3d

Try out Galaxy Nexus

439f216

Merge remote-tracking branch 'origin/main' into add-android-integrati…

495c218

…on-test-ci

Try different configurations

0627046

nilsreichardt added 2 commits October 1, 2022 23:32

Merge branch 'main' into add-android-integration-test-ci

73419a2

Merge branch 'main' into add-android-integration-test-ci

751d4c4

nilsreichardt added 5 commits February 22, 2023 22:05

Try to use flutter test

7d540fe

Add working dir

47d4bec

Increase timeout and add comments

a98418e

Uncomment api-level

cfcad3c

Use google_atd

afe26e2

nilsreichardt added 3 commits February 23, 2023 16:45

Use correct target

b74ab90

revert temp staff

ff0d2d8

Remove strategy

a95c991

nilsreichardt marked this pull request as ready for review February 23, 2023 22:58

Merge branch 'main' into add-android-integration-test-ci

817f919

Jonas-Sander approved these changes Feb 23, 2023

View reviewed changes

nilsreichardt added 3 commits February 24, 2023 07:49

Change order of emulator-options

ebfb5c9

Merge branch 'add-android-integration-test-ci' of https://github.com/…

8185a7d

…SharezoneApp/sharezone-app into add-android-integration-test-ci

Merge branch 'main' into add-android-integration-test-ci

c306e3e

nilsreichardt enabled auto-merge (squash) February 24, 2023 06:50

Re-trigger CI

63d1030

nilsreichardt merged commit 2be7dfb into main Feb 24, 2023

nilsreichardt deleted the add-android-integration-test-ci branch February 24, 2023 10:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate Android integration tests into CI pipeline #230

Integrate Android integration tests into CI pipeline #230

nilsreichardt commented May 17, 2022 •

edited

Loading

github-actions bot commented May 17, 2022 •

edited

Loading

Jonas-Sander left a comment

Jonas-Sander May 17, 2022

nilsreichardt May 18, 2022

nilsreichardt commented May 18, 2022

Jonas-Sander commented Sep 30, 2022

github-actions bot commented Feb 22, 2023 •

edited

Loading

nilsreichardt commented Feb 23, 2023

Sharezone-Admin commented Feb 23, 2023 •

edited

Loading

Jonas-Sander left a comment

Jonas-Sander Feb 23, 2023

nilsreichardt Feb 24, 2023

Jonas-Sander Feb 23, 2023

nilsreichardt Feb 24, 2023

Integrate Android integration tests into CI pipeline #230

Integrate Android integration tests into CI pipeline #230

Conversation

nilsreichardt commented May 17, 2022 • edited Loading

github-actions bot commented May 17, 2022 • edited Loading

Jonas-Sander left a comment

Choose a reason for hiding this comment

Jonas-Sander May 17, 2022

Choose a reason for hiding this comment

nilsreichardt May 18, 2022

Choose a reason for hiding this comment

nilsreichardt commented May 18, 2022

Jonas-Sander commented Sep 30, 2022

github-actions bot commented Feb 22, 2023 • edited Loading

nilsreichardt commented Feb 23, 2023

Sharezone-Admin commented Feb 23, 2023 • edited Loading

Jonas-Sander left a comment

Choose a reason for hiding this comment

Jonas-Sander Feb 23, 2023

Choose a reason for hiding this comment

nilsreichardt Feb 24, 2023

Choose a reason for hiding this comment

Jonas-Sander Feb 23, 2023

Choose a reason for hiding this comment

nilsreichardt Feb 24, 2023

Choose a reason for hiding this comment

nilsreichardt commented May 17, 2022 •

edited

Loading

github-actions bot commented May 17, 2022 •

edited

Loading

github-actions bot commented Feb 22, 2023 •

edited

Loading

Sharezone-Admin commented Feb 23, 2023 •

edited

Loading