Bump pytest from 7.4.4 to 8.3.2

[dependabot]

Bumps pytest from 7.4.4 to 8.0.0.

Release notes

Sourced from pytest's releases.

pytest 8.0.0 (2024-01-27)

See 8.0.0rc1 and 8.0.0rc2 for the full changes since pytest 7.4!

Bug Fixes

• #11842: Properly escape the reason of a skip <pytest.mark.skip ref>{.interpreted-text role="ref"} mark when writing JUnit XML files.
• #11861: Avoid microsecond exceeds 1_000_000 when using log-date-format with %f specifier, which might cause the test suite to crash.

8.0.0rc2

pytest 8.0.0rc2 (2024-01-17)

Improvements

• #11233: Improvements to -r for xfailures and xpasses:
  • Report tracebacks for xfailures when -rx is set.
  • Report captured output for xpasses when -rX is set.
  • For xpasses, add - in summary between test name and reason, to match how xfail is displayed.
• #11825: The pytest_plugin_registered{.interpreted-text role="hook"} hook has a new plugin_name parameter containing the name by which plugin is registered.

Bug Fixes

• #11706: Fix reporting of teardown errors in higher-scoped fixtures when using [--maxfail]{.title-ref} or [--stepwise]{.title-ref}.

• #11758: Fixed IndexError: string index out of range crash in if highlighted[-1] == "\n" and source[-1] != "\n". This bug was introduced in pytest 8.0.0rc1.

• #9765, #11816: Fixed a frustrating bug that afflicted some users with the only error being assert mod not in mods. The issue was caused by the fact that str(Path(mod)) and mod.__file__ don't necessarily produce the same string, and was being erroneously used interchangably in some places in the code.

  This fix also broke the internal API of PytestPluginManager.consider_conftest by introducing a new parameter -- we mention this in case it is being used by external code, even if marked as private.

pytest 8.0.0rc1 (2023-12-30)

See https://docs.pytest.org/en/latest/changelog.html#pytest-8-0-0rc1-2023-12-30 for the rendered changelog.

Breaking Changes

Old Deprecations Are Now Errors

• #7363: PytestRemovedIn8Warning deprecation warnings are now errors by default.

  Following our plan to remove deprecated features with as little disruption as possible, all warnings of type PytestRemovedIn8Warning now generate errors instead of warning messages by default.

  The affected features will be effectively removed in pytest 8.1, so please consult the deprecations{.interpreted-text role="ref"} section in the docs for directions on how to update existing code.

  In the pytest 8.0.X series, it is possible to change the errors back into warnings as a stopgap measure by adding this to your pytest.ini file:

  [pytest]

... (truncated)

Commits

• 478f823 Prepare release version 8.0.0
• 6085900 [8.0.x] fix: avoid rounding microsecond to 1_000_000 (#11863)
• 3b41c65 [8.0.x] Escape skip reason in junitxml (#11845)
• 747072a [8.0.x] Update docstring of scripts/generate-gh-release-notes.py (#11768)
• 011a475 Properly attach packages to the GH release notes (#11839) (#11840)
• 97960bd Merge pull request #11835 from pytest-dev/release-8.0.0rc2
• 6be0a3c Prepare release version 8.0.0rc2
• 44ffe07 Merge pull request #11837 from pytest-dev/backport-11836-to-8.0.x
• 14ecb04 [8.0.x] testing: temporarily disable test due to hypothesis issue
• 41c8dab Merge pull request #11831 from bluetech/backport-11825-to-8.0.x
• Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

[cooperlees]

O, this might been changes ... let's see

[cooperlees]

We are blocked until this plugin gets 8.0.0 support: pytest-dev/pytest-asyncio#737

[cooperlees]

Now seems test_delete_path has a closed event loop for it's second run of delete_path ... maybe separate test for it?

[dependabot]

A newer version of pytest exists, but since this PR has been edited by someone other than Dependabot I haven't updated it. You'll get a PR for the updated version as normal once this PR is merged.

[flyinghyrax]

I poked at this a bit yesterday and here are my observations so far:

1. The first call to delete_path inside test_delete_path succeeds because its dry run argument is True, so it doesn't try to access the event loop. If you step into this call with a debugger, the event loop attached to the storage plugin is already closed.

2. The failure happens when any of the following test modules execute before test_delete.py:

   • plugins/test_*
   • test_main.py
   • test_mirror.py
   • test_sync.py
   • test_verify.py

   But if test_delete.py runs first, it succeeds. e.g. this fails:

   pytest ./src/bandersnatch/tests/plugins/test_regex_name.py ./src/bandersnatch/tests/test_delete.py
   

   But this succeeds:

   pytest ./src/bandersnatch/tests/test_delete.py ./src/bandersnatch/tests/plugins/test_regex_name.py
   

3. If I add a function-scoped test fixture to the tests in test_delete.py that calls storage.storage_backend_plugins(clear_cache=True), i.e. destroys and recreates the storage plugins before each test, every test in the module fails with the same error (event loop being closed).

---

Python 3.12.7, pytest 8.3.2, pytest-asyncio 0.23.8

[flyinghyrax]

Executed each test module in combination with test_delete.py, with the other module run before test_delete:

pytest src/bandersnatch/tests/<MODULE> src/bandersnatch/tests/test_delete.py

Table Columns

• Runtime Error - did test_delete.py::test_delete_path fail with RuntimeError: Event loop is closed (yes/no)
• Deprecation Warning - did a call to asyncio.get_event_loop() trigger DeprecationWarning: There is no current event loop, and if so the test name and Bandersnatch module
  • Per https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_event_loop:

    Deprecated since version 3.12: Deprecation warning is emitted if there is no current event loop. In some future Python release this will become an error.

Module before test_delete.py	Runtime Error	Deprecation Warning
plugins/test_allowlist_name.py	yes	TestAllowListProject::test__filter__commented__out -> master.py:47
plugins/test_blocklist_name.py	yes	TestBlockListProject::test__filter__matches__package -> master.py:47
plugins/test_filename.py	yes	TestExcludePlatformFilter::test_exclude_platform -> master.py:47
plugins/test_latest_release.py	yes	TestLatestReleaseFilter::test_latest_releases_ensure_reusable -> master.py:47
plugins/test_metadata_plugins.py	yes	TestSizeProjectMetadataFilter::test__filter__size__only -> master.py:47
plugins/test_prerelease_name.py	yes	TestRegexReleaseFilter::test_plugin_filter_all -> master.py:47
plugins/test_regex_name.py	yes	TestRegexReleaseFilter::test_plugin_check_match -> master.py:47
plugins/test_storage_plugin_s3.py	-	-
plugins/test_storage_plugins.py	yes	TestFilesystemStoragePlugin::test_canonicalize_package -> storage.py:90
test_configuration.py	-	-
test_filter.py	-	-
test_main.py	yes	-
test_master.py	-	-
test_mirror.py	yes	test_mirror.py::test_mirror_loads_serial -> storage.py:90
test_package.py	-	test_package.py::test_package_update_... -> master.py
test_simple.py	-	test_simple.py::test_json_index_page -> storage.py:90
test_sync.py	yes	test_sync.py::test_sync_specific_packages -> master.py:47
test_utils.py	-	-
test_verify.py	yes	-

---

It also doesn't seem to be related to test_delete_path being the first test in the test_delete.py module. This run order fails:

PS> pytest --verbose .\src\bandersnatch\tests\test_verify.py  `
>> .\src\bandersnatch\tests\test_delete.py::test_delete_simple_page `
>> .\src\bandersnatch\tests\test_delete.py::test_delete_path

[flyinghyrax]

I noticed that in several other places where we fetch the current storage plugin, we were passing the clear_cache flag. On a lark, tried this and it does indeed fix the test:

--- i/src/bandersnatch/delete.py
+++ w/src/bandersnatch/delete.py
@@ -21,7 +21,7 @@ logger = logging.getLogger(__name__)


 async def delete_path(blob_path: Path, dry_run: bool = False) -> int:
-    storage_backend = next(iter(storage_backend_plugins()))
+    storage_backend = next(iter(storage_backend_plugins(clear_cache=True)))
     if dry_run:
         logger.info(f" rm {blob_path}")
         return 0

This doesn't seem like a practical solution to me, since delete_packages may create a large number of delete_path coroutines, and this would re-initialize the current storage plugin for every single one.

[flyinghyrax]

I was also seeing a few failures in test_main.py caused by the configuration singleton not being reset for those functions. They were missing a call to the 'setup' function used by other tests in the same module. I tried changing this to use pytest's usefixture instead and it worked:

diff --git i/src/bandersnatch/tests/test_main.py w/src/bandersnatch/tests/test_main.py
index 47fcdd1..3d55bbd 100644
--- i/src/bandersnatch/tests/test_main.py
+++ w/src/bandersnatch/tests/test_main.py
@@ -24,11 +24,14 @@ async def empty_dict(*args: Any, **kwargs: Any) -> dict:
     return {}


-def setup() -> None:
-    """simple setup function to clear Singleton._instances before each test"""
+@pytest.fixture
+def clear_config_singleton() -> None:
     Singleton._instances = {}


+pytestmark = pytest.mark.usefixtures("clear_config_singleton")
+
+
 def test_main_help(capfd: CaptureFixture) -> None:
     sys.argv = ["bandersnatch", "--help"]
     with pytest.raises(SystemExit):
@@ -103,7 +106,6 @@ def test_main_reads_config_values(mirror_mock: mock.MagicMock, tmpdir: Path) ->
 def test_main_reads_custom_config_values(
     mirror_mock: "BandersnatchMirror", logging_mock: mock.MagicMock, customconfig: Path
 ) -> None:
-    setup()
     conffile = str(customconfig / "bandersnatch.conf")
     sys.argv = ["bandersnatch", "-c", conffile, "mirror"]
     main(asyncio.new_event_loop())
@@ -114,7 +116,6 @@ def test_main_reads_custom_config_values(
 def test_main_throws_exception_on_unsupported_digest_name(
     customconfig: Path,
 ) -> None:
-    setup()
     conffile = str(customconfig / "bandersnatch.conf")
     parser = configparser.ConfigParser()
     parser.read(conffile)

I think I should be able to put together a clean commit for this in a bit. But I need to double check that this isn't a problem I caused in my local working tree, since it doesn't seem related to Pytest 7x -> 8x.

[cooperlees]

@flyinghyrax did this elsewhere

[dependabot]

OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting @dependabot ignore this major version or @dependabot ignore this minor version. You can also ignore all major, minor, or patch releases for a dependency by adding an ignore condition with the desired update_types to your config file.

If you change your mind, just re-open this PR and I'll resolve any conflicts on it.