Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Flaky test] TestController_RotateCertificates #4224

Closed
antoninbas opened this issue Sep 13, 2022 · 0 comments · Fixed by #4241
Closed

[Flaky test] TestController_RotateCertificates #4224

antoninbas opened this issue Sep 13, 2022 · 0 comments · Fixed by #4241
Assignees
@antoninbas
Labels
area/test Issues or PRs related to unit and integration tests. kind/bug Categorizes issue or PR as related to a bug. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test.

Comments

@antoninbas
Copy link
Contributor

I have seen the test fail in CI (Unit test (ubuntu-latest)), with the following logs:

2022-09-13T22:07:23.5780808Z W0913 22:07:12.163091   23569 env.go:88] Environment variable POD_NAMESPACE not found
2022-09-13T22:07:23.5782372Z W0913 22:07:12.163183   23569 env.go:126] Failed to get Pod Namespace from environment. Using "kube-system" as the Antrea Service Namespace
2022-09-13T22:07:23.5783304Z E0913 22:07:12.560524   23569 ipsec_certificate_controller.go:261] "Verifying current certificate configurations failed" err="certificate and key pair is nil"
2022-09-13T22:07:23.5784002Z I0913 22:07:12.560579   23569 ipsec_certificate_controller.go:268] "Start rotating IPsec certificate"
2022-09-13T22:07:23.5784733Z E0913 22:07:12.744620   23569 ipsec_certificate_controller.go:261] "Verifying current certificate configurations failed" err="CA certificate is empty"
2022-09-13T22:07:23.5785408Z I0913 22:07:12.744669   23569 ipsec_certificate_controller.go:268] "Start rotating IPsec certificate"
2022-09-13T22:07:23.5786114Z E0913 22:07:13.428822   23569 ipsec_certificate_controller.go:261] "Verifying current certificate configurations failed" err="certificate and key pair is nil"
2022-09-13T22:07:23.5787053Z I0913 22:07:13.428877   23569 ipsec_certificate_controller.go:268] "Start rotating IPsec certificate"
2022-09-13T22:07:23.5787759Z E0913 22:07:13.749592   23569 ipsec_certificate_controller.go:261] "Verifying current certificate configurations failed" err="certificate and key pair is nil"
2022-09-13T22:07:23.5788477Z I0913 22:07:13.749644   23569 ipsec_certificate_controller.go:268] "Start rotating IPsec certificate"
2022-09-13T22:07:23.5789721Z I0913 22:07:14.922615   23569 ipsec_certificate_controller.go:471] "Created new certificate and key for IPSec" cert="/tmp/antrea-ipsec-test169962986/fake-node-1-9ebb1176b6.crt" key="/tmp/antrea-ipsec-test169962986/fake-node-1-9ebb1176b6.key"
2022-09-13T22:07:23.5791031Z I0913 22:07:14.927151   23569 ipsec_certificate_controller.go:223] "Calculated certificate rotation deadline" expiration="2022-09-14 22:07:13 +0000 UTC" deadline="2022-09-14 16:51:58.879519679 +0000 UTC"
2022-09-13T22:07:23.5791855Z I0913 22:07:14.927249   23569 ipsec_certificate_controller.go:383] "Updating OVS configurations for IPsec certificates"
2022-09-13T22:07:23.5792491Z I0913 22:07:14.927841   23569 ipsec_certificate_controller.go:383] "Updating OVS configurations for IPsec certificates"
2022-09-13T22:07:23.5793143Z I0913 22:07:15.240443   23569 ipsec_certificate_controller.go:411] "Starting AntreaAgentIPsecCertificateController"
2022-09-13T22:07:23.5793850Z E0913 22:07:15.240612   23569 ipsec_certificate_controller.go:261] "Verifying current certificate configurations failed" err="certificate and key pair is nil"
2022-09-13T22:07:23.5794487Z I0913 22:07:15.240671   23569 ipsec_certificate_controller.go:268] "Start rotating IPsec certificate"
2022-09-13T22:07:23.5795589Z I0913 22:07:16.608027   23569 ipsec_certificate_controller.go:471] "Created new certificate and key for IPSec" cert="/tmp/antrea-ipsec-test1602412469/fake-node-1-20f4979105.crt" key="/tmp/antrea-ipsec-test1602412469/fake-node-1-20f4979105.key"
2022-09-13T22:07:23.5796733Z I0913 22:07:16.609320   23569 ipsec_certificate_controller.go:223] "Calculated certificate rotation deadline" expiration="2022-09-13 22:07:25 +0000 UTC" deadline="2022-09-13 22:07:22.030147099 +0000 UTC"
2022-09-13T22:07:23.5797415Z I0913 22:07:16.609408   23569 ipsec_certificate_controller.go:383] "Updating OVS configurations for IPsec certificates"
2022-09-13T22:07:23.5797999Z I0913 22:07:22.031616   23569 ipsec_certificate_controller.go:268] "Start rotating IPsec certificate"
2022-09-13T22:07:23.5798584Z --- FAIL: TestController_RotateCertificates (8.33s)
2022-09-13T22:07:23.5799367Z     ipsec_certificate_controller_test.go:382: Sign CSR "fake-node-1-qzfspc5x" successfully
2022-09-13T22:07:23.5800191Z     ipsec_certificate_controller_test.go:382: Sign CSR "fake-node-1-f8pbjlnq" successfully
2022-09-13T22:07:23.5800761Z     ipsec_certificate_controller_test.go:317: 
2022-09-13T22:07:23.5801690Z         	Error Trace:	/home/runner/work/antrea/antrea/pkg/agent/controller/ipseccertificate/ipsec_certificate_controller_test.go:317
2022-09-13T22:07:23.5802446Z         	Error:      	"7s" is not less than or equal to "6.656622822s"
2022-09-13T22:07:23.5803757Z         	Test:       	TestController_RotateCertificates
2022-09-13T22:07:23.5804772Z I0913 22:07:23.261391   23569 ipsec_certificate_controller.go:471] "Created new certificate and key for IPSec" cert="/tmp/antrea-ipsec-test1602412469/fake-node-1-d174243e57.crt" key="/tmp/antrea-ipsec-test1602412469/fake-node-1-d174243e57.key"
2022-09-13T22:07:23.5805834Z I0913 22:07:23.262802   23569 ipsec_certificate_controller.go:223] "Calculated certificate rotation deadline" expiration="2022-09-13 22:07:32 +0000 UTC" deadline="2022-09-13 22:07:30.793340975 +0000 UTC"
2022-09-13T22:07:23.5806446Z I0913 22:07:23.262865   23569 ipsec_certificate_controller.go:383] "Updating OVS configurations for IPsec certificates"
2022-09-13T22:07:23.5807015Z I0913 22:07:23.262923   23569 ipsec_certificate_controller.go:435] "Shutting down AntreaAgentIPsecCertificateController"
2022-09-13T22:07:23.5808078Z I0913 22:07:23.263657   23569 ipsec_certificate_controller.go:223] "Calculated certificate rotation deadline" expiration="2022-05-20 10:00:00 +0000 UTC" deadline="2022-05-20 08:03:18.342424885 +0000 UTC"
2022-09-13T22:07:23.5809006Z I0913 22:07:23.263795   23569 ipsec_certificate_controller.go:223] "Calculated certificate rotation deadline" expiration="2022-05-30 00:00:00 +0000 UTC" deadline="2022-05-28 03:14:17.302095332 +0000 UTC"
2022-09-13T22:07:23.5809401Z FAIL
@antoninbas antoninbas added kind/bug Categorizes issue or PR as related to a bug. area/test Issues or PRs related to unit and integration tests. labels Sep 13, 2022
@antoninbas antoninbas self-assigned this Sep 20, 2022
antoninbas added a commit to antoninbas/antrea that referenced this issue Sep 20, 2022
@antoninbas
Fix flaky TestController_RotateCertificates unit test
80547bc 
The difference in CSR CreationTimestamp can be lower than 7s because the
resolution of the certificate's NotAfter field is at the second level,
which means the provided expiration time will be truncated.

Fixes antrea-io#4224

Signed-off-by: Antonin Bas <[email protected]>
@antoninbas antoninbas mentioned this issue Sep 20, 2022
Merged
@antoninbas antoninbas added the kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. label Sep 20, 2022
antoninbas added a commit that referenced this issue Sep 21, 2022
@antoninbas
Fix flaky TestController_RotateCertificates unit test (#4241)
b36a884 
The difference in CSR CreationTimestamp can be lower than 7s because the
resolution of the certificate's NotAfter field is at the second level,
which means the provided expiration time will be truncated.

Fixes #4224

Signed-off-by: Antonin Bas <[email protected]>
xliuxu pushed a commit to xliuxu/antrea that referenced this issue Dec 14, 2022
@antoninbas @xliuxu
Fix flaky TestController_RotateCertificates unit test
c113f45 
The difference in CSR CreationTimestamp can be lower than 7s because the
resolution of the certificate's NotAfter field is at the second level,
which means the provided expiration time will be truncated.

Fixes antrea-io#4224

Signed-off-by: Antonin Bas <[email protected]>
tnqn pushed a commit that referenced this issue Dec 14, 2022
@antoninbas @tnqn
Fix flaky TestController_RotateCertificates unit test
0c689c1 
The difference in CSR CreationTimestamp can be lower than 7s because the
resolution of the certificate's NotAfter field is at the second level,
which means the provided expiration time will be truncated.

Fixes #4224

Signed-off-by: Antonin Bas <[email protected]>
heanlan pushed a commit to heanlan/antrea that referenced this issue Mar 29, 2023
@antoninbas @heanlan
Fix flaky TestController_RotateCertificates unit test (antrea-io#4241)
166369a 
The difference in CSR CreationTimestamp can be lower than 7s because the
resolution of the certificate's NotAfter field is at the second level,
which means the provided expiration time will be truncated.

Fixes antrea-io#4224

Signed-off-by: Antonin Bas <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antoninbas antoninbas

Labels
area/test Issues or PRs related to unit and integration tests. kind/bug Categorizes issue or PR as related to a bug. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix flaky TestController_RotateCertificates unit test antoninbas/antrea
1 participant
@antoninbas