Mark tests for async test suite more accurately #3180
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fselmo
changed the title
fselmo
changed the title
reedsa
reviewed
Jan 10, 2024
tox.ini
Outdated
| core: pytest {posargs:tests/core -k "not async"} | ||
| core_async: pytest {posargs:tests/core -k async} | ||
| core: pytest {posargs:tests/core -m "not asyncio" -k "not async"} | ||
| core_async: pytest {posargs:tests/core -k "asyncio or async"} |
There was a problem hiding this comment.
Should asyncio be part of the mark expression flag? I think this will find any test names containing asyncio or async (which in turn finds anything that would include asyncio I believe).
There was a problem hiding this comment.
Yep, good catch. I think it should be the opposite of the previous. Will change.
There was a problem hiding this comment.
I think this approach needs to be revised actually... I don't believe the -m and -k work well enough together. We may just have to run asyncio marked tests and the opposite as:
pytest tests/core -m asyncio: run tests that are actually asynchronous in naturepytest tests/core -m "not asyncio": run all other tests, including those that may involve async classes but aren't asynchronous in nature (e.g. testing some static, non-asynchronous, property of some asynchronous class)
fselmo
added a commit
to fselmo/web3.py
that referenced
this pull request
Jan 10, 2024
fselmo
added a commit
to fselmo/web3.py
that referenced
this pull request
Jan 10, 2024
fselmo
force-pushed
the
run-asyncio-tests-more-efficiently
branch
from
8ea55be to
b5d8fca
Compare
fselmo
added a commit
to fselmo/web3.py
that referenced
this pull request
Jan 10, 2024
fselmo
added a commit
to fselmo/web3.py
that referenced
this pull request
Jan 10, 2024
fselmo
added a commit
to fselmo/web3.py
that referenced
this pull request
Jan 10, 2024
fselmo
force-pushed
the
run-asyncio-tests-more-efficiently
branch
from
aa99744 to
493fa0a
Compare
- All async tests need the ``@pytest.mark.asyncio`` decorator to be picked up by pytest-asyncio. This already marks them as asyncio tests, so we should use the ``-m`` option to call them as well as looking for "async" in the test name with the``-k`` option. This is necessary since some async tests only test for properties and aren't actually asynchronous but we want to consider them as part of the async test suite (e.g. testing some attribute on the AsyncHttpProvider). - Reviewers, be sure to check that the selected number in one test run equals the deselected number in the other.
fselmo
force-pushed
the
run-asyncio-tests-more-efficiently
branch
from
493fa0a to
dbfe191
Compare
reedsa
approved these changes
Jan 10, 2024
I looked at the
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What was wrong?
All async tests need the
@pytest.mark.asynciodecorator to be picked up by pytest-asyncio. This already marks them as asyncio tests, so we should use the-moption to call them. Looking for "async" in the test name with the-koption caused us to miss some tests that we didn't name with "async" in them in the async test suite.Note to any reviewers: Be sure to check that the selected number in one test run equals the deselected number in the other.
Todo:
Cute Animal Picture