Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: adjust ByTestId misleading warnings; add guidelines about accessibilityHint #542

Merged
merged 4 commits into from
Sep 11, 2020

Conversation

thymikee
Copy link
Member

@thymikee thymikee commented Sep 9, 2020

Summary

This PR is inspired by the discussion happening here wix/Detox#2313.

There's no need for us to recommend users whether they should avoid ByTestId queries or not. In fact, I noticed that developers tend to take our recommendations seriously and, maybe due to lack of better ideas, use accessibilityHint the same way as testID, which is even worse for the users, as they get some rubbish developer metadata instead of actual feedback.

For reference, here's Apple guideline on how and when accessibilityHint should be used: https://developer.apple.com/documentation/objectivec/nsobject/1615093-accessibilityhint. I've adjusted our example accordingly, so it doesn't spread misleading information.

Test plan

Docs

@mdjastrzebski
Copy link
Member

IMO I would consider rewording these notes rather than removing them. Please compare to RTL's note:

In the spirit of the guiding principles, it is recommended to use this only after the other queries don't work for your use case. Using data-testid attributes do not resemble how your software is used and should be avoided if possible. That said, they are way better than querying based on DOM structure or styling css class names. Learn more about data-testids from the blog post "Making your UI tests resilient to change"

Source: https://testing-library.com/docs/dom-testing-library/api-queries#bytestid

Maybe we can add info that:

  1. testIds are generally useful in Detox testing
  2. accessability hint should be meaningful to the user (with link to Apple's docs)

Wdyt?

@LeoNatan
Copy link

LeoNatan commented Sep 9, 2020

On iOS, all accessibility properties are used by the accessibility system to present to the user, so all text in those comprises of user-facing, localized strings. From our experience, such strings are not a good fit for matching views:

  1. They are localized strings, so sometimes language and/or locale of the simulator on CI doesn't match the language and/or locale of the one writing tests; when testing different language/locale configurations, it would be impossible to test by text
  2. User facing text changes over time, and tests tend to break
  3. The chances of several elements having the same text or label (or hint for that matter) increase as the size of the application grows and the view hierarchy becomes more and more complex

The exception to this are accessibility identifiers, which are not used by the OS for anything other than matching. From experience, using those to give a unique across the app identifier to user elements is the best way to match elements uniquely. We understand that this may not be practical at all times, but we do try to steer our users (internally in the company and community) toward such an approach first, and then try different things if this is not possible.

@LeoNatan
Copy link

LeoNatan commented Sep 9, 2020

At any rate, I don't think any guideline should be advising people to outright avoid test identifiers.

@mdjastrzebski
Copy link
Member

@LeoNatan I do not advise against using test ids for end to end tests, if they are the most viable solution then fine.

I do however believe that for RNTL integration tests, test ids are kind of escape hatches, you should use them when other methods of finding elements are not good enough. And indeed there are many cases for using test ids: e.g. container elements, screens, etc but usually not text inputs, buttons, etc.

My reasoning follows from React Testing Library guiding principle:

The more your tests resemble the way your software is used, the more confidence they can give you.

The reason why using getByText, etc is preferred is that testID attributes do not resemble how the app is used. App users do not see testIDs while the see Text instances, TextInput placeholders, they can recognize accessibility roles (or at least should be), they can interact with accessibility hints when using voice-over, etc.

Addressing your points:

  1. [Localized strings]: In RNTL integration tests, the tests run in Jest, not in simulator, and you have control over the default language (probably usually English). Finally your use will see these strings, so why not check for them.
  2. [Text change over time]: I consider this an advantage, as e.g if you change text input placeholder from username to email then you probably should change your test logic as well. Besides, if it's only text change these kind of changes are very fast to fix.
  3. [Repeating labels]: That's true, I've encountered that couple of times myself, you can mitigate it be using testID or by scoping queries to container element (probably got by using testID) and then using within helper. The benefit is that you test what user actually sees vs hidden identifier.

Regarding replacing testId with accessibilityHint: this does not mean we should put test ids into accessiblity hint. Obviously this would create awful voice-over user experience. However, we might put some meaningful user-audible text and use that for our element queries.

@thymikee thymikee changed the title docs: remove byTestId misleading warnings docs: adjust ByTestId misleading warnings; add guidelines about accessibilityHint Sep 10, 2020
Copy link
Member

@mdjastrzebski mdjastrzebski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I think the new notes are more explanatory then before.

website/docs/Queries.md Show resolved Hide resolved
@thymikee thymikee merged commit a0faf69 into master Sep 11, 2020
@thymikee thymikee deleted the docs/testid-warnings branch September 11, 2020 11:18
@Kjaer
Copy link

Kjaer commented Oct 21, 2020

That's brilliant!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants