Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TestDefaultResultTracker_StartMinimumRequests_NoFailingRequests flakyness #491

Conversation

pracucci
Copy link
Contributor

@pracucci pracucci commented Feb 20, 2024

What this PR does:

In this CI run we've seen TestDefaultResultTracker_StartMinimumRequests_NoFailingRequests being flaky:

--- FAIL: TestDefaultResultTracker_StartMinimumRequests_NoFailingRequests (20.67s)
    replication_set_tracker_test.go:294: 
        	Error Trace:	/drone/src/ring/replication_set_tracker_test.go:294
        	Error:      	Max difference between 750 and 695 allowed is 50, but difference was 55
        	Test:       	TestDefaultResultTracker_StartMinimumRequests_NoFailingRequests
        	Messages:   	expected roughly even distribution of requests across all instances, but got [{[] 769} {[] 772} {[] 764} {[] 695}]
FAIL

The test checks whether each instance got roughly the same number of requests. The selection of instances getting the "minimum requests" is generated randomly. Since there's no perfect random distribution, the actual difference may be greater than the 50 delta configured for the assertion.

In this PR I've just increased the delta to reduce the likelihood of flaky executions.

Which issue(s) this PR fixes:

N/A

Checklist

  • Tests updated
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@pracucci pracucci marked this pull request as ready for review February 20, 2024 13:03
@pracucci pracucci merged commit d6732a3 into main Feb 20, 2024
3 checks passed
@pracucci pracucci deleted the fix-TestDefaultResultTracker_StartMinimumRequests_NoFailingRequests branch February 20, 2024 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants