Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make all tests discoverable on Linux #1399

Merged
merged 1 commit into from
May 5, 2022

Conversation

glbrntt
Copy link
Collaborator

@glbrntt glbrntt commented May 4, 2022

Motivation:

Test discovery on Linux relies on source code analysis. Tests in
subclasses which do not explicitly redeclare each test are missed. As a
result, a number of tests are not run on Linux.

Modifications:

  • Explicitly redeclare tests in subclasses.

Result:

Motivation:

Test discovery on Linux relies on source code analysis. Tests in
subclasses which do not explicitly redeclare each test are missed. As a
result, a number of tests are not run on Linux.

Modifications:

- Explicitly redeclare tests in subclasses.

Result:

- Resolves grpc#1392
@glbrntt glbrntt added the semver/none No version bump required. label May 4, 2022
@glbrntt
Copy link
Collaborator Author

glbrntt commented May 4, 2022

I diffed the output of swift test --list-tests on macOS and linux to determine which tests were missing.

I was hoping we might be able to use swift package dump-package to analyse functions in test modules and diff them with the list of tests (so we could check if any tests are missing as part of CI) but dump-package does not include test targets.

@simonjbeaumont
Copy link
Collaborator

OOI, what's going to stop this from diverging when a new test is added to the base class? Do you think it's worth adding a CI check for the diff you performed manually for this PR?

@Lukasa
Copy link
Collaborator

Lukasa commented May 4, 2022

I'm +1 with @simonjbeaumont: I'd love to see a tool written for this that we can reuse on our other repos.

@glbrntt
Copy link
Collaborator Author

glbrntt commented May 4, 2022

Oh I totally agree, a tool would be great and that's what I looked at doing first:

I was hoping we might be able to use swift package dump-package to analyse functions in test modules and diff them with the list of tests (so we could check if any tests are missing as part of CI) but dump-package does not include test targets.

If you have any other ideas I'm all ears!

@glbrntt
Copy link
Collaborator Author

glbrntt commented May 4, 2022

One option is parsing all the test files and attempting to build up a graph of test classes and analysing them that way. Not ideal but probably workable.

@glbrntt glbrntt requested a review from Lukasa May 5, 2022 10:27
@glbrntt glbrntt merged commit 555b004 into grpc:main May 5, 2022
@glbrntt glbrntt deleted the gb-fix-test-discovery branch May 5, 2022 13:36
glbrntt added a commit to glbrntt/grpc-swift that referenced this pull request May 6, 2022
bimawa pushed a commit to StreamLayer/grpc-swift that referenced this pull request Jun 17, 2022
Motivation:

Test discovery on Linux relies on source code analysis. Tests in
subclasses which do not explicitly redeclare each test are missed. As a
result, a number of tests are not run on Linux.

Modifications:

- Explicitly redeclare tests in subclasses.

Result:

- Resolves grpc#1392
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver/none No version bump required.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Several test cases do not run on Linux
3 participants