Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI builds for iOS 13 are flaking #29

Closed
NickEntin opened this issue Oct 7, 2020 · 4 comments
Closed

CI builds for iOS 13 are flaking #29

NickEntin opened this issue Oct 7, 2020 · 4 comments
Labels
bug Something isn't working as expected

Comments

@NickEntin
Copy link
Collaborator

The CI build on iOS 13 seems to be flaking on a few specific tests:

testTabBars()
- Item B: 3 items. Tab. 2 of 4.
+ Item B. Tab. 2 of 4.
- Item C: A. Tab. 3 of 4.
+ Item C. Tab. 3 of 4.

testStepper()
- Decrement. Button.
+ minus. Button.
- Increment. Button.
+ plus. Button.

testStepperAtMin()
- Decrement. Dimmed. Button.
+ minus. Dimmed. Button.
- Increment. Button.
+ plus. Button.

This started after bumping the minimum iOS version to 12.0 (#27). I've seen this flake twice now.

@NickEntin
Copy link
Collaborator Author

Based on some testing in #30, it looks like this is specifically caused by bumping the framework's minimum version from iOS 10 to iOS 12. More specifically, the change to the sample app's deployment target is not what caused the flakes, and therefore I do not expect this is an issue that consumers are currently seeing.

This means that we need to dig into the root cause of these flakes before moving forward with the 0.4.0 release.

@NickEntin
Copy link
Collaborator Author

From #31, it looks like changing the minimum supported iOS version to 11.0 is a viable option.

@NickEntin NickEntin added the bug Something isn't working as expected label Oct 16, 2020
@NickEntin NickEntin linked a pull request Oct 16, 2020 that will close this issue
@NickEntin
Copy link
Collaborator Author

My theory for the testStepper and testStepperAtMin cases is that _UIStepperButton isn't marking itself as an element, so VoiceOver is treating the image views instead the buttons as the elements. The accessibilityLabels for those image views are "minus" and "plus".

@NickEntin
Copy link
Collaborator Author

From some debugging in #36, it seems like the problem is in the test target, not the framework itself. I think at this point we're okay to move forward with other PRs and continue with the 0.4 release, since it seems that any other change to the test target will fix the tests.

I'm going to go ahead and close this issue for now. If we see more flaky tests on a subsequent PR, let's reopen this and continue investigating.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment