Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p2p: remove TestP2PThreeNodesEvenDist #5736

Merged
merged 1 commit into from
Sep 13, 2023

Conversation

iansuvak
Copy link
Contributor

@iansuvak iansuvak commented Sep 12, 2023

Summary

Currently the TestP2PThreeNodesEvenDist fails pretty regularly. The theory here is that part of the problem is that P2P stream connections take longer to setup than the wsNetwork and that because of that nodes get into a bad state with consensus trying to move too quickly. The errors on the CI runs for these tests indicate that nodes are trying to catchup via gossip protocol but these attempts fail on timeouts.

After further investigation it looks like this behavior is not specific to P2P but happens on the websocket network as well. 50~60% of the time the test takes significantly longer locally but still succeeds. The failure on the CircleCI side is likely just different resourcing leading to slower execution and slower recovery from the bad state that nodes get into. Removing the test for now.

FWIW P2P does pass the PartitionRecovery tests where this network template is used on the websocket network side.

Test Plan

This is a test only change. We want to see whether this change stops the existing test from being so flaky while discussing options to make it more stable and support the faster lambda.

This only removes a flaky test, it should hopefully bring nightly builds back into a stable state.

@codecov
Copy link

codecov bot commented Sep 12, 2023

Codecov Report

Merging #5736 (844588b) into master (527d2c5) will increase coverage by 1.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5736      +/-   ##
==========================================
+ Coverage   54.49%   55.56%   +1.07%     
==========================================
  Files         476      476              
  Lines       66853    66853              
==========================================
+ Hits        36429    37149     +720     
+ Misses      27876    27180     -696     
+ Partials     2548     2524      -24     

see 41 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@iansuvak iansuvak added p2p Work related to the p2p project test Improves testing of existing code Skip-Release-Notes labels Sep 12, 2023
cce
cce previously approved these changes Sep 12, 2023
@iansuvak iansuvak changed the title p2p: don't shorten lambda for p2p e2e tests p2p: remove TestP2PThreeNodesEvenDist Sep 13, 2023
@cce cce requested a review from algorandskiy September 13, 2023 14:43
@algorandskiy algorandskiy merged commit ecb9afb into algorand:master Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p2p Work related to the p2p project Skip-Release-Notes test Improves testing of existing code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants