Improve file and helper selection

[novemberborn]

I've run out of time today and can't finish this PR:

• Update documentation
• Figure out why reporter integration tests changed
• Check behavior on Windows, especially with watch mode
• Ensure files starting with an underscore are recognized as source files, not changed test files (watch mode)
• Test on some actual projects
• Support helpers globs (see Ava does not transpile helpers! #1319) filed Support helpers globs #2105
• Reference the open issues this solves
• Does this solve Change in source file doesn't trigger test run in watch mode when started with test selection #1264?
• Does this solve ava incorrectly ignores test files when the project itself is contained in node_modules directory #1173?
• Does this solve Ava rerunning all tests when changing single test file, when using .ts files. #2085?
• Ensure our linter plugin recognizes the new syntax filed Support new file and helper selection eslint-plugin-ava#238
• "Global" excludes which ensures globs are excluded by the watcher too filed Allow global exclusion patterns to be configured #2106

Here's a quick summary of changes. I can expand on rationale later:

• Require globs to match files, not directories
• files and sources must be an array of one or more patterns, not a string
• Only accept files via the CLI, not globs or directories
• Upgrade globby
• At this stage, only test files starting with _ are treated as helpers.
  Helper globs should be made configurable.
• It's no longer possible to override an exclusion pattern. We may
  reintroduce this at a later stage

Fixes #1228
Fixes #909
Fixes #1418
Fixes #1793
Fixes #1501
Fixes #736
Fixes #1540
Fixes #1288
Fixes #1173
Fixes #2085

[novemberborn]

I can expand on rationale later

I'm thinking it's actually quite confusing that files may match directories. It's easier to understand you need to "specify glob patterns that match test files". Similarly, it's easier to understand that the CLI arguments must be actual file paths rather than glob patterns.

Any helper matched as a test file will still be treated as a helper. For now that's just files starting with an underscore. But before we can ship this as a 2.0 we need to make it possible to specify glob patterns for helper files. These then can be tuned to user's project setup. Any test file that also matches the helper patterns will be considered a helper, not a test file.

If you disable the Babel pipeline, AVA won't need to find helper files, just exclude test files that are helpers.

With this PR, test files are still found based on the configured extensions. It just takes a lot less code.

You can specify solely exclusion patterns, which will be applied together with the default patterns.

Overriding the default exclusion patterns is no longer possible. If this becomes a problem we can introduce a setting to customize the default exclusion patterns.

My plan is to get this working and then ship it as a 2.0 beta.

[novemberborn]

With regards to our ESLint plugin, my idea's to expose ava/eslint-helper which can provide the necessary functionality, without needing to expose the full AVA configuration to the plugin, or re-implement functionality there.

[sindresorhus]

But before we can ship this as a 2.0 we need to make it possible to specify glob patterns for helper files.

I was hoping for v2 we could drop the concept of helpers (which users generally find surprising and awkward) in favor just transpiling everything.

[novemberborn]

I was hoping for v2 we could drop the concept of helpers (which users generally find surprising and awkward) in favor just transpiling everything.

That's not going to happen in the next say two weeks.

Even then, presumably we need the ability to compile tests differently from sources, and this helper work will still be necessary.

Hopefully with these changes helpers are easier to understand though. It'll be test files that start with an underscore. Everything else will be opt-in.

[sindresorhus]

Does this solve #1264?

Does it?

[novemberborn]

Does it?

No, it's a Chokidar limitation.