Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable differing_test_runners health check #108886

Merged
merged 6 commits into from
Sep 4, 2023
Merged

Disable differing_test_runners health check #108886

merged 6 commits into from
Sep 4, 2023

Conversation

sobolevn
Copy link
Member

@sobolevn sobolevn commented Sep 4, 2023 •
edited
Loading

@sobolevn sobolevn requested a review from hugovk September 4, 2023 17:17
@AlexWaygood
Copy link
Member

AlexWaygood commented Sep 4, 2023 •
edited
Loading

Should we maybe also pin the version of hypothesis we're installing in CI here? https://github.com/python/cpython/blob/main/.github/workflows/build.yml#L434

It seems fairly likely that Hypothesis could add other health checks in the future that would cause our CI to randomly start failing. And it's just generally good practice to pin test dependencies as well, so that we can easily reproduce errors that are happening in CI.

@sobolevn
Copy link
Member Author

sobolevn commented Sep 4, 2023

@dependabot and @sobolevn will bump hypothesis from time to time 👍

AlexWaygood
AlexWaygood reviewed Sep 4, 2023
View reviewed changes
Tools/requirements-tests.txt Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe requirements-hypothesis.txt? Tools/requirements-tests.txt feels too similar to the existing Tools/requirements-dev.txt file we already have (and I think it's unlikely we'd use the same file for test dependencies that aren't related to the Hypothesis CI job).

I'm not even sure the file belongs in the Tools/ directory... maybe it should just be a top-level file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependabot already knows about this dir and Hypothesis is clearly a tool :)
So, let's leave it there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could move them all into some other directory later, maybe .github/dependabot/, but let's get the CI back to green first.

Copy link
Member

@AlexWaygood AlexWaygood Sep 4, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, Tools/ still doesn't feel ideal to me, but I don't feel strongly about where they should go 👍

AlexWaygood
AlexWaygood approved these changes Sep 4, 2023
View reviewed changes
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! Will wait a little bit in case Hugo/Zac/Ezio want to chime in, though :)

@Zac-HD
Copy link
Contributor

Zac-HD commented Sep 4, 2023

I'm a big fan of pinning your transitive dependencies too with e.g. the pip-tools package, so long as they're updated regularly - hypothesis itself has an automated weekly PR for that.

We most recently added a new health check in January 2021, but I'd pin anyway. For example, a while ago Astropy's CI started failing because a new Hypothesis release was better at finding numerical bugs, and while you do want to find out about them (updates) you don't want them to break everyone's regression tests (pins). See also this blog post.

@sobolevn
Copy link
Member Author

sobolevn commented Sep 4, 2023 •
edited
Loading

Two transitive deps that are installed in CI are: attrs and sortedcontainers. I think we can skip pinning them (the first regression here can prove me wrong).

Thanks for your help 👍

@hugovk hugovk enabled auto-merge (squash) September 4, 2023 18:08
@hugovk hugovk added the needs backport to 3.12 bug and security fixes label Sep 4, 2023
@hugovk hugovk merged commit 6ead5bd into python:main Sep 4, 2023
@miss-islington
Copy link
Contributor

Thanks @sobolevn for the PR, and @hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-108887 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 bug and security fixes label Sep 4, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 4, 2023
@sobolevn @miss-islington
Disable differing_test_runners health check (pythonGH-108886)
28d8f80 
(cherry picked from commit 6ead5bd)

Co-authored-by: Nikita Sobolev <[email protected]>
Yhg1s pushed a commit that referenced this pull request Sep 4, 2023
@miss-islington @sobolevn
[3.12] Disable differing_test_runners health check (GH-108886) (#10…
bce7791 
…8887)

Disable `differing_test_runners` health check (GH-108886)
(cherry picked from commit 6ead5bd)

Co-authored-by: Nikita Sobolev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlexWaygood AlexWaygood AlexWaygood approved these changes

@hugovk hugovk hugovk approved these changes

@ezio-melotti ezio-melotti Awaiting requested review from ezio-melotti ezio-melotti is a code owner

Assignees
No one assigned
Labels
skip issue skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@sobolevn @AlexWaygood @Zac-HD @miss-islington @bedevere-bot @hugovk