Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: use fresh AVD from benchmark tests #157

Merged
merged 1 commit into from
Jul 3, 2024

Conversation

CAMOBAP
Copy link
Collaborator

@CAMOBAP CAMOBAP commented Jul 3, 2024

Caching AVD for Android is a questionable practice due to the complexity of emulator setup and unwanted states that should not be stored between CI runs. As a result, this leads to unstable tests on CI.

We definitely should avoid it for benchmarking, as the environment must be as clean as possible between runs.

@CAMOBAP CAMOBAP requested review from e271828- and DSergiu July 3, 2024 11:26
@CAMOBAP CAMOBAP self-assigned this Jul 3, 2024
Copy link

github-actions bot commented Jul 3, 2024

Diffuse report:

OLD: sdk-main.aar
NEW: sdk-pr.aar

 AAR      │ old       │ new       │ diff 
──────────┼───────────┼───────────┼──────
      jar │  51.7 KiB │  51.7 KiB │  0 B 
 manifest │     411 B │     411 B │  0 B 
      res │  78.3 KiB │  78.3 KiB │  0 B 
    other │     954 B │     954 B │  0 B 
──────────┼───────────┼───────────┼──────
    total │ 131.3 KiB │ 131.3 KiB │  0 B 

 JAR     │ old │ new │ diff      
─────────┼─────┼─────┼───────────
 classes │  37 │  37 │ 0 (+0 -0) 
 methods │ 507 │ 507 │ 0 (+0 -0) 
  fields │ 162 │ 162 │ 0 (+0 -0)

Copy link

github-actions bot commented Jul 3, 2024

Benchmark report:

Test name Time ms. (median) Allocations (median)
com.hcaptcha.sdk.HCaptchaBenchmarkTest.EMULATOR_UNLOCKED_benchmarkInvisibleVerification +331.89 -5
com.hcaptcha.sdk.HCaptchaBenchmarkTest.EMULATOR_UNLOCKED_benchmarkInvisibleVerificationColdRun +358.68 +106
com.hcaptcha.sdk.HCaptchaBenchmarkTest.EMULATOR_UNLOCKED_benchmarkInvisibleSetup -0.50 +96
com.hcaptcha.sdk.HCaptchaDebugInfoTest.EMULATOR_UNLOCKED_benchmarkDebugInfo -8.01 +556
com.hcaptcha.sdk.HCaptchaDebugInfoTest.EMULATOR_UNLOCKED_benchmarkDebugSys +20.28 +39
com.hcaptcha.sdk.HCaptchaWebViewHelperTest.EMULATOR_UNLOCKED_benchmarkWebViewLoad +517.26 +95

@CAMOBAP CAMOBAP merged commit 124caaa into main Jul 3, 2024
12 checks passed
@CAMOBAP CAMOBAP deleted the chore/use-fresh-avd-on-benchmark branch July 3, 2024 12:40
Copy link

sonarqubecloud bot commented Jul 4, 2024

CAMOBAP added a commit that referenced this pull request Jul 16, 2024
* ci: skip sonarqube step for PR from form #109

* ci: use fresh AVD from benchmark tests (#157)

* refactor: build gradle (#161)

* chore: split big sdk/build.gradle to several applied gradle sripts

* chore: upload all outputs and reports on benchmark failure

* ci: move timeout restoriction to step level and reduce it to 20

* debug: enable artifact upload for success bench

* fix: ignore HCaptchaWebViewHelperTest.benchmarkWebViewLoad benchmark

* ci: skip sonarqube step for PR from form #109

* ci: migrate to own check-user-permission action

* fix: bad prior merge

* fix: add value to outputs of check-user-permission action
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants