Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix prefect-dask test suite #16920

Merged
merged 7 commits into from
Jan 31, 2025
Merged

Fix prefect-dask test suite #16920

merged 7 commits into from
Jan 31, 2025

Conversation

desertaxle
Copy link
Member

@desertaxle desertaxle commented Jan 31, 2025

prefect-dask tests are failing on main. This PR fixes that by removing unnecessary fixtures like the scoped event loop and use_hosted_api_server (prefect_test_harness takes care of that), reusing Dask clusters where possible, and updating two tests to run with only a single task runner because they aren't dependent on the task runner configuration.

@desertaxle desertaxle changed the title Increase connect timeout in tests Fix prefect-dask test suite Jan 31, 2025
@desertaxle desertaxle marked this pull request as ready for review January 31, 2025 18:29
@desertaxle desertaxle added the development Tech debt, refactors, CI, tests, and other related work. label Jan 31, 2025
Copy link
Collaborator

@zzstoatzz zzstoatzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

Comment on lines -17 to -47


@pytest.fixture(scope="session")
def event_loop(request):
"""
Redefine the event loop to support session/module-scoped fixtures;
see https://github.com/pytest-dev/pytest-asyncio/issues/68
When running on Windows we need to use a non-default loop for subprocess support.
"""
if sys.platform == "win32":
asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())

policy = asyncio.get_event_loop_policy()

loop = policy.new_event_loop()

# configure asyncio logging to capture long running tasks
asyncio_logger = logging.getLogger("asyncio")
asyncio_logger.setLevel("WARNING")
asyncio_logger.addHandler(logging.StreamHandler())
loop.set_debug(True)
loop.slow_callback_duration = 0.25

try:
yield loop
finally:
loop.close()

# Workaround for failures in pytest_asyncio 0.17;
# see https://github.com/pytest-dev/pytest-asyncio/issues/257
policy.set_event_loop(loop)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋

@desertaxle desertaxle merged commit 5b5725a into main Jan 31, 2025
25 checks passed
@desertaxle desertaxle deleted the debug-prefect-dask-tests branch January 31, 2025 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Tech debt, refactors, CI, tests, and other related work.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants