Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CryptoSDK phased rollout feature #7374

Merged
merged 2 commits into from
Feb 22, 2023
Merged

CryptoSDK phased rollout feature #7374

merged 2 commits into from
Feb 22, 2023

Conversation

Anderas
Copy link
Contributor

@Anderas Anderas commented Feb 15, 2023

Implement CryptoSDKFeature which combines three different ways to enable the rust-based Crypto SDK:

  • manually enabling in the Labs section of settings
  • automatically enabling if remote feature flag in PostHog is enabled (requires PostHog update to 2.0.0)
  • automatically enabling if local feature flag in phased rollout is enabled

Given that the migration to CryptoSDK is quite a big change we want to be as careful as possible when automatically converting users to it. Ideally the rate of convertion would be controlled completely remotely, so that we do not have to make new app releases to increase or stop the rollout. We can use PostHog for this, however only a subset of users will have PostHog and analytics enabled, meaning there is a limit to the number of users that can be targetted with the remote feature.

To address the remainder of users we rely on local deterministic bucketing based on userId, where each bucket represents a single percentage. To determine whether a particular user is included in the current stage of rollout, we interpret their bucket as number between 0% and 100% and compare to the hardcoded targetPercentage. To increase target percentage a new app release is required.

@Anderas Anderas force-pushed the andy/crypto_feature branch from 36c1971 to 1ac1e2a Compare February 15, 2023 13:46
@Anderas Anderas requested review from a team and aringenbach and removed request for a team February 15, 2023 13:47
@codecov
Copy link

codecov bot commented Feb 15, 2023

Codecov Report

Base: 12.11% // Head: 12.16% // Increases project coverage by +0.04% 🎉

Coverage data is based on head (40005fe) compared to base (be93ff5).
Patch coverage: 77.14% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #7374      +/-   ##
===========================================
+ Coverage    12.11%   12.16%   +0.04%     
===========================================
  Files         1640     1642       +2     
  Lines       162133   162195      +62     
  Branches     66587    66605      +18     
===========================================
+ Hits         19643    19729      +86     
+ Misses      141843   141820      -23     
+ Partials       647      646       -1     
Flag Coverage Δ
unittests 6.02% <77.14%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
Riot/Modules/Application/LegacyAppDelegate.m 14.55% <0.00%> (ø)
Riot/Modules/Settings/SettingsViewController.m 0.00% <0.00%> (ø)
Riot/Experiments/CryptoSDKFeature.swift 79.62% <79.62%> (ø)
Riot/Experiments/Experiment.swift 94.11% <94.11%> (ø)
Config/CommonConfiguration.swift 88.40% <100.00%> (+1.60%) ⬆️
Riot/Experiments/PhasedRolloutFeature.swift 100.00% <100.00%> (ø)
Riot/Modules/Analytics/Analytics.swift 13.00% <100.00%> (+3.58%) ⬆️
...iot/Modules/Analytics/PostHogAnalyticsClient.swift 71.87% <100.00%> (+6.30%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Comment on lines 63 to 65
XCTAssertEqual(variants.count, 5)
XCTAssertTrue(variants.contains(0))
XCTAssertTrue(variants.contains(1))
XCTAssertTrue(variants.contains(2))
XCTAssertTrue(variants.contains(3))
XCTAssertTrue(variants.contains(4))
XCTAssertFalse(variants.contains(5))
Copy link
Contributor

@aringenbach aringenbach Feb 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing these kind of tests could fail under some super unlucky circumstances ? (probably almost never with 10000)
But maybe testing something like variants.isSubset(of: Set<UInt>(0, 1, 2, 3, 4)) makes sense here ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea probabilities are always tricky to test deterministically, even the variant.count >= 2 assertion could theoretically fail, though at 10000 samples highly unlikely. Swapped to your suggestion

Copy link
Contributor

@aringenbach aringenbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small suggestion, otherwise LGTM

@Anderas Anderas force-pushed the andy/crypto_feature branch from 7561f6c to 40005fe Compare February 22, 2023 11:25
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@Anderas Anderas merged commit a845488 into develop Feb 22, 2023
@Anderas Anderas deleted the andy/crypto_feature branch February 22, 2023 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants