-
Notifications
You must be signed in to change notification settings - Fork 999
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(ci): Add support for test_timeout option #3379
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following up on a recent conversation, is the configuration parameter needed? The alternative would be to have a static large timeout for all tests. I would assume we optimize running-time for the happy case only. I would expect most pull requests to pass the tests (happy case) and only a small subset to fail. As a developer I care about the majority of my pull requests to be tested quickly. I don't care so much that CI runs longer on buggy pull requests, given that they are rare.
I am not deeply involved here. Please feel free to ignore. Change looks good to me. Feel free to add the send-it
label.
This pull request has merge conflicts. Could you please resolve them @MarcoPolo? 🙏 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think having the timeout configurable is overall a reasonable feature. IMO, one thing that we should keep in the back of our mind is that ideally, all of the different test binaries that we will be building should have the same or at least consistent interface. That would allow us to build shared abstractions between them and makes things overall easier to understand.
Agreed. Right now this is still in the evolving phase, but soon I'll write up what this interface is.
This is not needed. We have the reasonable default of 10s. This lets us extend this timeout explicitly when we need to. I rather make it explicit that NodeJS and the Browser tests need more time. It would be surprising if Go/Rust hit these timeouts (Maybe the CI worker is overloaded or affected by a noisy neighbor?). This is really a small point, and doesn't have much bearing either way. |
359dff8
to
cbf84c4
Compare
Rebased off master. Sorry for doing this right after your approval @thomaseizinger |
No worries. Unless there were conflicts, there is no need as the merge bot will update the PR! :) |
Description
Node and browser tests take a little bit longer to spin up in the interop tests. The multidim-interop can tell the tests to increase the timeout when testing against these targets, but the test itself needs to use the passed in value.
This updates the test code to handle the new test_timeout option. See libp2p/test-plans#107
Notes
Links to any relevant issues
Open Questions
Change checklist