Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): Add support for test_timeout option #3379

Merged
merged 2 commits into from
Jan 25, 2023
Merged

Conversation

MarcoPolo
Copy link
Contributor

Description

Node and browser tests take a little bit longer to spin up in the interop tests. The multidim-interop can tell the tests to increase the timeout when testing against these targets, but the test itself needs to use the passed in value.

This updates the test code to handle the new test_timeout option. See libp2p/test-plans#107

Notes

Links to any relevant issues

Open Questions

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • A changelog entry has been made in the appropriate crates

Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on a recent conversation, is the configuration parameter needed? The alternative would be to have a static large timeout for all tests. I would assume we optimize running-time for the happy case only. I would expect most pull requests to pass the tests (happy case) and only a small subset to fail. As a developer I care about the majority of my pull requests to be tested quickly. I don't care so much that CI runs longer on buggy pull requests, given that they are rare.

I am not deeply involved here. Please feel free to ignore. Change looks good to me. Feel free to add the send-it label.

@mergify
Copy link
Contributor

mergify bot commented Jan 24, 2023

This pull request has merge conflicts. Could you please resolve them @MarcoPolo? 🙏

Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having the timeout configurable is overall a reasonable feature. IMO, one thing that we should keep in the back of our mind is that ideally, all of the different test binaries that we will be building should have the same or at least consistent interface. That would allow us to build shared abstractions between them and makes things overall easier to understand.

@MarcoPolo
Copy link
Contributor Author

same or at least consistent interface

Agreed. Right now this is still in the evolving phase, but soon I'll write up what this interface is.

Following up on a recent conversation, is the configuration parameter needed?

This is not needed. We have the reasonable default of 10s. This lets us extend this timeout explicitly when we need to. I rather make it explicit that NodeJS and the Browser tests need more time. It would be surprising if Go/Rust hit these timeouts (Maybe the CI worker is overloaded or affected by a noisy neighbor?).

This is really a small point, and doesn't have much bearing either way.

@MarcoPolo
Copy link
Contributor Author

Rebased off master. Sorry for doing this right after your approval @thomaseizinger

@thomaseizinger
Copy link
Contributor

Rebased off master. Sorry for doing this right after your approval @thomaseizinger

No worries. Unless there were conflicts, there is no need as the merge bot will update the PR! :)

@mergify mergify bot merged commit 1b45d5e into master Jan 25, 2023
@mergify mergify bot deleted the marco/add-timeout branch January 25, 2023 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants