-
Notifications
You must be signed in to change notification settings - Fork 913
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid coupling tests with hardcoded test CA certs #6086
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Yihao Wang <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #6086 +/- ##
==========================================
- Coverage 48.35% 48.11% -0.25%
==========================================
Files 666 668 +2
Lines 54880 55163 +283
==========================================
+ Hits 26537 26541 +4
- Misses 26618 26896 +278
- Partials 1725 1726 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/assign
Thank you @y1hao ! Excellent!
Yes! You are right, I tested it on my side and all tests passed after reverting the copied certificate. Shall we deprecate (remove) the copied certificate? I think the right practice is like what you did in this PR, which generates a fresh certificate from the test server. Even the rest tests don't go through the test server, but they need a cert, why not give them a fresh one? My concern about keeping the copied certificate is that sometimes the developers imitate the legacy tests when they want to introduce new test cases. They may meet the same issue again. |
Signed-off-by: Yihao Wang <[email protected]>
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@RainbowMango Yeah, I agree that the best practice would be to use a fresh cert for each test case. But the concern here is that generating a new certificate is a heavy process, both in terms of compute resource consumed and the amount of code needed. For this reason, I think people usually don't create a new cert on the fly for testing, instead, they just use a pre-generated one. When we retrieve the cert from the test server, it looks like that we are getting a fresh one for each test case, but in fact, we are just doing some decoding to retrieve the exact same cert every time, which is the previously copied one. If you compare the cert retrieved from the test server and the cert we copied from Go's Similarly, k8s also has a pre-generated test certificate to use in tests: https://github.com/kubernetes/kubernetes/blob/f015fd66ce95d02cd66efc263eb9e5441b42a17d/hack/testdata/ca/ca.crt So, I've updated this PR to use a new pre-generated certificate to replace the copied one, and documented the command used to generate it. This is the same practice as used in Go and k/k.
With the new cert, if someone incorrectly used the pre-generated test cert when they should retrieve the cert from the test server, their test will fail so they can fix this before merging. I've also moved the code to retrieve the cert from the test server to a helper method, to help people understand its purpose. |
What type of PR is this?
/kind flake
What this PR does / why we need it:
As we discovered in #6066, 2 test cases in
membercluster_client_test.go
depend directly on a hardcoded test CA certificate, which was duplicated from Go'snet/http/internal
package. This can cause these tests to fail when a new version of Go has regenerated that internal certificate. The test server constructed with Go'shttptest
package already exposes a method to retrieve its trusted CA. This PR updates the tests to use that method to retrieve the CA cert directly from the test server.Other test cases in this file don't go through a test server, so they can use any cert. They don't need to be updated.
Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?: