Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E tests #1143

Closed
wants to merge 61 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
9972f2b
Test e2e
stefan-niedermann Oct 29, 2021
5f90ab3
Add sample test
stefan-niedermann Oct 29, 2021
964ea2f
Move e2e script into workflow
stefan-niedermann Oct 29, 2021
1f521bf
Login flow
stefan-niedermann Oct 29, 2021
03337f1
Add sleep
stefan-niedermann Oct 29, 2021
8c40b69
Fix package name of downloaded NC apk
stefan-niedermann Oct 29, 2021
e7715e5
Configure NC account via adb
stefan-niedermann Oct 29, 2021
5def0a0
another try - should fail
stefan-niedermann Oct 29, 2021
ad1751e
Be more explicit
stefan-niedermann Oct 30, 2021
8c9852c
Only run connectedDevDebugAndroidTest
stefan-niedermann Oct 30, 2021
9ff88ed
Use --health-cmd instead of sleepUntilReady.sh
stefan-niedermann Oct 30, 2021
abe5dd2
Try new e2e test using UI Automator
stefan-niedermann Oct 31, 2021
d56b890
Remove Files app setup from workflow
stefan-niedermann Oct 31, 2021
074b42b
Make and store screenshots
stefan-niedermann Oct 31, 2021
4886e87
Add more screenshots
stefan-niedermann Oct 31, 2021
1cb08af
Rename screenshots
stefan-niedermann Oct 31, 2021
9c5ade7
Refactoring
stefan-niedermann Oct 31, 2021
e2a9753
List files
stefan-niedermann Oct 31, 2021
a862e7a
upload everything
stefan-niedermann Oct 31, 2021
0628510
Upload reports as described in ...
stefan-niedermann Oct 31, 2021
9b8f799
Try pulling screenshots from runner machine
stefan-niedermann Oct 31, 2021
d13b8a7
Try pulling screenshots from runner machine
stefan-niedermann Oct 31, 2021
628e783
Fix test creds
stefan-niedermann Oct 31, 2021
741853c
Use shell script as own file
stefan-niedermann Oct 31, 2021
4195772
Try another syntax
stefan-niedermann Oct 31, 2021
f3d6553
More verbose
stefan-niedermann Oct 31, 2021
7bad0ca
Fix yml syntax
stefan-niedermann Oct 31, 2021
dbbaa52
Upload screenshots folder
stefan-niedermann Oct 31, 2021
30dd49f
Upload screenshots folder
stefan-niedermann Oct 31, 2021
f715c10
adb ls
stefan-niedermann Oct 31, 2021
ea189ef
More output
stefan-niedermann Oct 31, 2021
0f1923b
Do not adb shell ls recursively
stefan-niedermann Oct 31, 2021
3ba3ee5
Refactor e2e test
stefan-niedermann Nov 1, 2021
b35e378
Limit Logcat to info and higher
stefan-niedermann Nov 1, 2021
bfec6ea
Try to make usage of AVD caches
stefan-niedermann Nov 1, 2021
efa326d
Increase matrix multipliers
stefan-niedermann Nov 1, 2021
435f3ee
Fix typo in nextcloud:production image
stefan-niedermann Nov 1, 2021
8041889
Make docker filter more generous
stefan-niedermann Nov 1, 2021
51ccf6f
Make docker filter more generous
stefan-niedermann Nov 1, 2021
c4df126
Wait longer for Nextcloud app getting installed
stefan-niedermann Nov 1, 2021
4f48972
Remove copy_screenshots.sh
stefan-niedermann Nov 1, 2021
e84eae8
Do not fail when logcat can not be retrieved
stefan-niedermann Nov 1, 2021
b0994dd
Remove installed apps before e2e
stefan-niedermann Nov 1, 2021
4593dcb
Allow failing apk removal
stefan-niedermann Nov 1, 2021
4eaab54
Provocate test failure
stefan-niedermann Nov 1, 2021
683906f
Watch fail
stefan-niedermann Nov 2, 2021
3f3d6ae
Try to fix again
stefan-niedermann Nov 2, 2021
878cd9e
Use matrix for unit tests
stefan-niedermann Nov 2, 2021
c84b813
Remove ls
stefan-niedermann Nov 2, 2021
46a6944
Remove || true to make instrumented tests actually fail
stefan-niedermann Nov 2, 2021
e2ee15c
Restore successful tests
stefan-niedermann Nov 2, 2021
e59bbc4
[Should fail] Use connectedDevDebugAndroidTest flavor
stefan-niedermann Nov 2, 2021
f52b9c2
[Should success] Use connectedDevDebugAndroidTest flavor
stefan-niedermann Nov 2, 2021
2bf4993
Remove screenshots
stefan-niedermann Nov 2, 2021
c2906cf
[Should fail] Refactoring
stefan-niedermann Nov 2, 2021
9dec888
[Should success] Refactoring
stefan-niedermann Nov 2, 2021
2f74d85
[Should fail] Clean up
stefan-niedermann Nov 2, 2021
1f177c3
[Should success] Clean up
stefan-niedermann Nov 2, 2021
9b192c0
[Should fail] Readd adb logcat -c
stefan-niedermann Nov 2, 2021
4465cfc
[Should success] Readd adb logcat -c
stefan-niedermann Nov 2, 2021
b7d1aab
[Should fail] Readd adb logcat -c
stefan-niedermann Nov 2, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/workflows/android-e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
on: [push]

jobs:
setup_nextcloud:
runs-on: ubuntu-latest
name: Run e2e test
strategy:
matrix:
api-level: [ 24, 25, 26, 27, 28, 29 ]
nextcloud-version: [ 'nextcloud:latest', 'nextcloud:stable', 'nextcloud:production' ]
services:
nextcloud:
image: ${{ matrix.nextcloud-version }}
env:
SQLITE_DATABASE: db.sqlite
NEXTCLOUD_ADMIN_USER: Test
NEXTCLOUD_ADMIN_PASSWORD: Test
ports:
- 8080:80
options: >-
--health-cmd "curl GET 'http://Test:Test@localhost:80/ocs/v2.php/apps/serverinfo/api/v1/info' -f -H 'OCS-APIRequest: true' || exit 1"
--health-interval 1s
--health-timeout 2s
--health-retries 10
--health-start-period 3s
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Enable Deck via Docker Exec
run: |
docker exec `docker ps -f 'name=_nextcloud' -l -q` bash -c 'runuser -u www-data -- php occ app:install deck'

- name: Fetch capabilities
run: |
curl -X GET 'http://Test:Test@localhost:8080/ocs/v2.php/cloud/capabilities?format=json' -H 'OCS-APIRequest: true' | jq

##########################
# AVD CACHING START #
##########################

- name: Gradle cache
uses: actions/cache@v2
with:
path: |
~/.gradle/caches
~/.gradle/wrapper
key: gradle-${{ runner.os }}-${{ hashFiles('**/*.gradle*') }}-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties') }}-${{ hashFiles('**/buildSrc/**/*.kt') }}

- name: AVD cache
uses: actions/cache@v2
id: avd-cache
with:
path: |
~/.android/avd/*
~/.android/adb*
key: avd-${{ matrix.api-level }}

- name: Create AVD and generate snapshot for caching
if: steps.avd-cache.outputs.cache-hit != 'true'
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: ${{ matrix.api-level }}
force-avd-creation: false
emulator-options: -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: true
script: echo "Generated AVD snapshot for caching."

##########################
# AVD CACHING END #
##########################

- name: Run e2e tests
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: ${{ matrix.api-level }}
force-avd-creation: false
emulator-options: -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
disable-animations: true
script: |
adb shell pm uninstall -k --user 0 com.nextcloud.android.beta || true
adb shell pm uninstall -k --user 0 it.niedermann.nextcloud.deck.dev || true
wget -q https://download.nextcloud.com/android/dev/latest.apk
adb install latest.apk
sleep 10s
adb logcat -c || true
./gradlew connectedDevDebugAndroidTest
5 changes: 4 additions & 1 deletion .github/workflows/android.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,14 @@ jobs:
test:
name: Unit tests
runs-on: ubuntu-latest
strategy:
matrix:
flavor: [ 'testDevDebugUnitTest', 'testFdroidReleaseUnitTest', 'testPlayReleaseUnitTest' ]
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Unit tests
run: bash ./gradlew test
run: bash ./gradlew ${{ matrix.flavor }}

codeql:
name: CodeQL security scan
Expand Down
4 changes: 4 additions & 0 deletions app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -126,4 +126,8 @@ dependencies {
testImplementation 'org.mockito:mockito-core:4.0.0'
testImplementation 'androidx.test:core:1.4.0'
testImplementation 'androidx.arch.core:core-testing:2.1.0'

// Instrumented tests
androidTestImplementation 'androidx.test:runner:1.4.0'
androidTestImplementation 'androidx.test.uiautomator:uiautomator:2.2.0'
}
122 changes: 122 additions & 0 deletions app/src/androidTest/java/it/niedermann/nextcloud/deck/ui/E2ETest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
package it.niedermann.nextcloud.deck.ui;

import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation;

import android.content.Intent;
import android.webkit.WebView;
import android.widget.Button;
import android.widget.EditText;

import androidx.annotation.NonNull;
import androidx.test.uiautomator.By;
import androidx.test.uiautomator.UiDevice;
import androidx.test.uiautomator.UiObjectNotFoundException;
import androidx.test.uiautomator.UiSelector;
import androidx.test.uiautomator.Until;

import org.junit.After;
import org.junit.Before;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class E2ETest {

private UiDevice mDevice;

private static final int TIMEOUT = 30_000;

private static final String APP_NEXTCLOUD = "com.nextcloud.android.beta";
private static final String APP_DECK = "it.niedermann.nextcloud.deck.dev";

private static final String SERVER_URL = "http://localhost:8080";
private static final String SERVER_USERNAME = "Test";
private static final String SERVER_PASSWORD = "Test";

@Before
public void before() {
mDevice = UiDevice.getInstance(getInstrumentation());
}

@After
public void after() {
mDevice.pressHome();
}

@Test
public void test_00_configureNextcloudAccount() throws UiObjectNotFoundException {
launch(APP_NEXTCLOUD);

final var loginButton = mDevice.findObject(new UiSelector().text("Log in"));
loginButton.waitForExists(TIMEOUT);
loginButton.click();

mDevice.findObject(new UiSelector().focused(true)).setText(SERVER_URL);
mDevice.pressEnter();
mDevice.findObject(new UiSelector().text("Log in")).click();
mDevice.wait(Until.findObject(By.clazz(WebView.class)), TIMEOUT);

final var usernameInput = mDevice.findObject(new UiSelector()
.instance(0)
.className(EditText.class));
usernameInput.waitForExists(TIMEOUT);
usernameInput.setText(SERVER_USERNAME);

final var passwordInput = mDevice.findObject(new UiSelector()
.instance(1)
.className(EditText.class));
passwordInput.waitForExists(TIMEOUT);
passwordInput.setText(SERVER_PASSWORD);

mDevice.findObject(new UiSelector().text("Log in")).click();
mDevice.findObject(new UiSelector().text("Grant access")).click();
}

@Test
public void test_01_importAccountIntoDeck() throws UiObjectNotFoundException {
launch(APP_DECK);

final var accountButton = mDevice.findObject(new UiSelector()
.instance(0)
.className(Button.class));
accountButton.waitForExists(TIMEOUT);
accountButton.click();

final var radioAccount = mDevice.findObject(new UiSelector()
.clickable(true)
.instance(0));
radioAccount.waitForExists(TIMEOUT);
radioAccount.click();

final var okButton = mDevice.findObject(new UiSelector().text("OK"));
okButton.waitForExists(TIMEOUT);
okButton.click();

final var allowButton = mDevice.findObject(new UiSelector().text("Allow"));
allowButton.waitForExists(TIMEOUT);
allowButton.click();

final var welcomeText = mDevice.findObject(new UiSelector().description("Filter"));
welcomeText.waitForExists(TIMEOUT);
}

@Test
public void test_02_verifyCardsPresent() throws UiObjectNotFoundException {
launch(APP_DECK);

final var taskCard = mDevice.findObject(new UiSelector()
.textContains("Taask 3"));
taskCard.waitForExists(TIMEOUT);
System.out.println("Found: " + taskCard.getText());
}

private void launch(@NonNull String packageName) {
final var context = getInstrumentation().getContext();
context.startActivity(context
.getPackageManager()
.getLaunchIntentForPackage(packageName)
.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TASK));
mDevice.wait(Until.hasObject(By.pkg(packageName).depth(0)), TIMEOUT);
}
}