Register our tests in Flutter's "customer tests" suite #239
Comments
chrisbobbe
added a commit
to chrisbobbe/zulip-flutter
that referenced
this issue
Jul 24, 2023
… tests Along with 2f0f469, this is a reminder that it'd be good to get our repo added to Flutter's "customer tests" suite: zulip#239. It's also another reminder among several that it'd be good to pin a precise version of Flutter: zulip#15. Discussion: https://chat.zulip.org/#narrow/stream/243-mobile-team/topic/flutter.3A.20Failing.20tests/near/1615254 Related: zulip#15 Related: zulip#239
gnprice
pushed a commit
to chrisbobbe/zulip-flutter
that referenced
this issue
Jul 25, 2023
…tests Along with 2f0f469, this is a reminder that it'd be good to get our repo added to Flutter's "customer tests" suite: zulip#239. It's also another reminder among several that it'd be good to pin a precise version of Flutter: zulip#15. Discussion: https://chat.zulip.org/#narrow/stream/243-mobile-team/topic/flutter.3A.20Failing.20tests/near/1615254 Related: zulip#15 Related: zulip#239
gnprice
added a commit
to gnprice/zulip-flutter
that referenced
this issue
May 21, 2024
See comment for rationale. In particular this is required for adding our tests to Flutter's "customer tests", zulip#239: flutter/tests@a047b7a
chrisbobbe
pushed a commit
that referenced
this issue
May 22, 2024
See comment for rationale. In particular this is required for adding our tests to Flutter's "customer tests", #239: flutter/tests@a047b7a
|
This is done! The following PR in flutter/tests is now merged: |
This was referenced Jun 28, 2024
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Flutter upstream has a scheme for Flutter users' test suites to be integrated as part of Flutter's own CI, in the "customer tests" suite:
https://github.com/flutter/tests
This has the effect that a Flutter change which breaks those tests is treated as a "breaking change", the same category as a change which breaks Google's own internal tests:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
That in turn sets a higher bar for being worth the breakage and calls for more effort from the upstream author of the change to help developers migrate across it.
We should register for this scheme, now that we're out of the experimental phase and are committed to Flutter. The main prerequisite is that we'll need to have some appropriate CI scripts in the first place:
(The two major barriers that I suspect are the reasons most Flutter developers don't participate are (a) if your code isn't open-source, then it gets logistically complicated to do so, and (b) it necessarily requires you to be keeping up with Flutter
main/master, rather than only upgrading on stable releases. But we're open-source and are already keeping up with Flutter main, which we value because it lets us make upstream changes and promptly benefit from them.)I believe there are two occasions so far where this would have flagged an upstream change as breaking:
The text was updated successfully, but these errors were encountered: