Sonarqube including coverage on each PR. #6111

michaelkaye · 2022-05-20T16:35:43Z

Changes:

remove a lot of post-pr.yml tests (integration and sonarqube) as these run on each PR now
Leave behind the UI test as that is not part of 'code coverage'
Ignore a number of tests that were failing during the testing process - sonar coverage only works when tests run fine; so ensure we only have fine tests.
- NB: some of these tests may run fine locally, or may be able to be reintroduced with some tweaks; however at this point I'm explicitly ignoring any test that has failed, so we can start having a reliable test run - i know this is removing some probably good tests, and that is an explicit goal.
Update configuration to use a new sonarcloud.io project for uploads; this enables the PR notifications to be set correctly.

Once merged, we'll want to update the list of required checks before merging is OK; at the moment some of the checks are not being run or have been combined; this is global across the project and needs to occur after merging to develop.

After merge I suggest everyone updates their branches against latest develop before attempting a PR merge.

A concern might be usage of the macos runner to get the integration tests running each time. There's not much we can do about that at present.

michaelkaye · 2022-05-25T12:02:04Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs 0 Vulnerabilities 0 Security Hotspots 0 Code Smells

No Coverage information 0.0% Duplication

Unsure why this has no coverage marked on it yet; coverage is available in the sonarcloud.io UI.

michaelkaye · 2022-05-25T12:08:16Z

Though, given the whole macos thing; I might attempt to use a linux runner and see if it reliably fails, or if the failure rate that caused us to use the macos runner was related to tests that we have since ignored.

michaelkaye · 2022-05-25T12:09:03Z

"Build android tests" is now covered by generic tests - we run them as well as build them now.

"Run unit tests" is now covered by "Runs all tests" - we combined them with the integration tests.

michaelkaye · 2022-05-25T12:09:55Z

Test coverage (even with all the ignored tests) is now at 10% ish - a big improvement on the ~2% we were earlier reporting.

michaelkaye · 2022-05-25T12:17:13Z

coverage.gradle

@@ -18,11 +21,13 @@ def initializeReport(report, projects, classExcludes) {
            switch (project) {
                case { project.plugins.hasPlugin("com.android.application") }:
                    androidClassDirs.add("${project.buildDir}/tmp/kotlin-classes/gplayDebug")
+                    androidSourceDirs.add("${project.buildDir}/generated/source/kapt/gplayDebug")


Without this we have some generated files with coverage percentages but no source to look at. Maybe we should ignore them from the coverage instead?

I would be for ignoring all generated code (since we're technically not the author~)

Can be a separate PR after clearing our exclusions with the team

…age.

michaelkaye · 2022-05-25T12:55:21Z

Though, given the whole macos thing; I might attempt to use a linux runner and see if it reliably fails, or if the failure rate that caused us to use the macos runner was related to tests that we have since ignored.

Failed at the first attempt.

bmarty

A few remarks

bmarty · 2022-05-25T12:55:32Z

.github/workflows/tests.yml

@@ -12,73 +12,57 @@ env:
    -Porg.gradle.parallel=false

 jobs:
-  # Build Android Tests
-  build-android-tests: 
-    name: Build Android Tests 


Can we just disable this job, in case we need to easily enable it again for some reason?

I can leave it in but commented out; but it will be in github history anyway ?

Have left it in as commented out code.

OK, thanks. Yes, still visible in the git history, but a bit more hidden, so probably quite lost.
I was wondering if something like this exists: if: never() to disable a job.

bmarty · 2022-05-25T12:56:56Z

build.gradle

-        property "sonar.projectName", "Element-Android"
-        property "sonar.projectKey", "im.vector.app.android"
+        property "sonar.projectName", "element-android"
+        property "sonar.projectKey", "vector-im_element-android"


So the Sonar URL has changed? If yes, there are 3 lines to update in the file README.md

Have replaced with newly generated links - the existing ones seemed to be legacy versions

ouchadam · 2022-05-27T11:52:25Z

I'm not sure why these tasks are getting stuck 🤔

ouchadam · 2022-05-27T11:57:00Z

closed/reopened to see if the actions list will resync

bmarty

LGTM, can be merged.

…ichaelk/sonarqube_fixes

sonarqubecloud · 2022-06-06T22:53:20Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

michaelkaye · 2022-06-07T08:42:59Z

I'm not sure why these tasks are getting stuck thinking

They're stuck because they're an expected result (from the repository required tests) but are not being generated (the tests all run in one go rather than separately)

michaelkaye · 2022-06-07T08:43:49Z

Admin-merging through the missing "build android tests" and "run unit tests" as they're now covered by "runs all tests" which handles both unit and connected tests.

michaelkaye force-pushed the michaelk/sonarqube_fixes branch 2 times, most recently from 2f50c6d to 5e3046c Compare May 25, 2022 11:33

Ignore all tests that have failed during testing.

c59c5cf

michaelkaye force-pushed the michaelk/sonarqube_fixes branch from 5e3046c to 02797a1 Compare May 25, 2022 11:39

michaelkaye changed the title ~~DO NOT MERGE: sonarqube GHA testing~~ Sonarqube including coverage on each PR. May 25, 2022

element-hq deleted a comment from sonarqubecloud bot May 25, 2022

michaelkaye marked this pull request as ready for review May 25, 2022 12:09

michaelkaye commented May 25, 2022

View reviewed changes

michaelkaye mentioned this pull request May 25, 2022

DO NOT MERGE: What happens without macos #6150

Closed

Update to build scripts to enable sonarqube reporting including cover…

ba109a4

…age.

michaelkaye requested a review from ouchadam May 25, 2022 12:56

michaelkaye force-pushed the michaelk/sonarqube_fixes branch from 02797a1 to ba109a4 Compare May 25, 2022 12:57

bmarty reviewed May 25, 2022

View reviewed changes

michaelkaye added 3 commits May 25, 2022 14:03

Update README.md with latest links generated from sonarcloud site.

40e4c4c

Add commented out version of building integration tests for reference.

fd5b709

Merge branch 'develop' into michaelk/sonarqube_fixes

9f4f3f4

ouchadam approved these changes May 27, 2022

View reviewed changes

triggering actions

b0f4c87

ouchadam closed this May 27, 2022

ouchadam reopened this May 27, 2022

bmarty approved these changes May 30, 2022

View reviewed changes

ouchadam added 2 commits May 30, 2022 09:37

temporarily ignoring flaky test when ran on the CI

ead189a

Merge branch 'develop' of github.com:vector-im/element-android into m…

d18e7ad

…ichaelk/sonarqube_fixes

ouchadam force-pushed the michaelk/sonarqube_fixes branch from e46ea39 to d18e7ad Compare May 30, 2022 09:07

michaelkaye added 4 commits June 6, 2022 17:08

Retry ALL THE THINGS, and always send sonarqube

56608d5

We still might be flaky; retry entire test run if so.

e9cb417

Fix and/&& mixup

2332499

Fix ktlint error

c2ac07b

michaelkaye merged commit 10125f4 into develop Jun 7, 2022

michaelkaye deleted the michaelk/sonarqube_fixes branch June 7, 2022 08:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sonarqube including coverage on each PR. #6111

Sonarqube including coverage on each PR. #6111

michaelkaye commented May 20, 2022 •

edited

Loading

michaelkaye commented May 25, 2022

michaelkaye commented May 25, 2022

michaelkaye commented May 25, 2022

michaelkaye commented May 25, 2022

michaelkaye May 25, 2022

ouchadam May 25, 2022 •

edited

Loading

michaelkaye commented May 25, 2022

bmarty left a comment

bmarty May 25, 2022

michaelkaye May 25, 2022 •

edited

Loading

bmarty May 30, 2022

bmarty May 25, 2022

michaelkaye May 25, 2022

ouchadam commented May 27, 2022

ouchadam commented May 27, 2022

bmarty left a comment

sonarqubecloud bot commented Jun 6, 2022

michaelkaye commented Jun 7, 2022

michaelkaye commented Jun 7, 2022

Sonarqube including coverage on each PR. #6111

Sonarqube including coverage on each PR. #6111

Conversation

michaelkaye commented May 20, 2022 • edited Loading

michaelkaye commented May 25, 2022

michaelkaye commented May 25, 2022

michaelkaye commented May 25, 2022

michaelkaye commented May 25, 2022

michaelkaye May 25, 2022

Choose a reason for hiding this comment

ouchadam May 25, 2022 • edited Loading

Choose a reason for hiding this comment

michaelkaye commented May 25, 2022

bmarty left a comment

Choose a reason for hiding this comment

bmarty May 25, 2022

Choose a reason for hiding this comment

michaelkaye May 25, 2022 • edited Loading

Choose a reason for hiding this comment

bmarty May 30, 2022

Choose a reason for hiding this comment

bmarty May 25, 2022

Choose a reason for hiding this comment

michaelkaye May 25, 2022

Choose a reason for hiding this comment

ouchadam commented May 27, 2022

ouchadam commented May 27, 2022

bmarty left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Jun 6, 2022

michaelkaye commented Jun 7, 2022

michaelkaye commented Jun 7, 2022

michaelkaye commented May 20, 2022 •

edited

Loading

ouchadam May 25, 2022 •

edited

Loading

michaelkaye May 25, 2022 •

edited

Loading