Fix unit test flaky for IPsec controller #4016
Conversation
Codecov Report
@@ Coverage Diff @@
## main #4016 +/- ##
==========================================
+ Coverage 61.51% 67.75% +6.24%
==========================================
Files 294 297 +3
Lines 43726 43820 +94
==========================================
+ Hits 26897 29691 +2794
+ Misses 14578 11806 -2772
- Partials 2251 2323 +72
|
| } | ||
| } | ||
| }() | ||
| go newFakeSigner(t, |
There was a problem hiding this comment.
The fake watcher registers their watches synchronously, so it should be impossible that a watcher missed an event of an object created after the watch started.
I suspect the issue is that WaitForCertificate missed the update event triggered by the signer because it uses a ListWatch, the fake clientset doesn't support watching from a specific resource version, so a fake ListWatch actually sends two individual requests, a List request and a subsequent Watch request. If the signer happens to sign the CSR in between the two requests, WaitForCertificate would never receive the update event.
This can be confirmed by adding a log to the signer to check when the failure happens, does the signer signs a CSR.
My previous comment about the defect of fake clientset:
antrea/pkg/agent/controller/noderoute/node_route_controller_test.go
Lines 101 to 103 in 4b788e7
There was a problem hiding this comment.
You are right and I misunderstood the goroutine in the Watch API. I think the issue did exist in WaitForCertificate when using the fake clientset. But I did not come up with a simple and clean solution for this issue. How about just waiting for one second before updating the CSR in the tests?
There was a problem hiding this comment.
I can't think of a better solution, I'm fine with fixing it in this way first. If it still fails in the future, perhaps we could updating the CSR multiple times to make sure the watcher will receive updates.
xliuxu
force-pushed
the
fix_ipsec_controller_ut_flaky
branch
2 times, most recently
from
9cb00ed to
4e7f1e9
Compare
pkg/agent/controller/ipseccertificate/ipsec_certificate_controller_test.go
Outdated
Show resolved
Hide resolved
Wait one second as the fake clientset doesn't support watching with specific resourceVersion. Otherwise the update event would be missed by the watcher used in csrutil.WaitForCertificate() if it happens to be generated in-between the List and Watch calls. Fixes: antrea-io#3851 Signed-off-by: Xu Liu <[email protected]>
xliuxu
force-pushed
the
fix_ipsec_controller_ut_flaky
branch
from
4e7f1e9 to
5251bd0
Compare
|
/skip-all |
Wait one second as the fake clientset doesn't support watching with specific resourceVersion. Otherwise the update event would be missed by the watcher used in csrutil.WaitForCertificate() if it happens to be generated in-between the List and Watch calls. Fixes: antrea-io#3851 Signed-off-by: Xu Liu <[email protected]>
Wait one second as the fake clientset doesn't support watching with specific resourceVersion.
Otherwise the update event would be missed by the watcher used in
csrutil.WaitForCertificate()if it happens to be generated in-between the List and Watch calls.
Fixes: #3851
Signed-off-by: Xu Liu [email protected]