Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce CRD size. #5089

Closed
wants to merge 2 commits into from
Closed

Conversation

blackpiglet
Copy link
Contributor

@blackpiglet blackpiglet commented Jul 4, 2022

The background of this change is Velero has some use cases that k8s cluster operated after proxy or air-gapped environment, so the network speed is limited. This makes Velero installation often failed with CRD creation timeout, and the biggest CRD of Velero is Restore, because the InitContainer hook contains PodSpec. Restore CRD size is nearly 2MB.

This is a proposal to reduce CRD size. Reduction is achieved by two modifications:

  • Remove all descriptions in CRDs.
  • Change Restore's InitContainter in Hook from corev1.PodSpec to runtime.RawExtension.

Not sure whether users and developers of Velero relied on the things changed. If anything is not acceptable, I'm open to change.

  1. remove descriptions in CRDs.
  2. Make the Restore hook.InitConatianer server side field pruing disable.
  3. Remove restore patch in update-generate-crd-code.sh.
  4. Modify related testcases.

Signed-off-by: Xun Jiang [email protected]

Thank you for contributing to Velero!

Please add a summary of your change

Does your change fix a particular issue?

Fixes #4916

Please indicate you've done the following:

  • Accepted the DCO. Commits without the DCO will delay acceptance.
  • Created a changelog file or added /kind changelog-not-required as a comment on this pull request.
  • Updated the corresponding documentation in site/content/docs/main.

@reasonerjt
Copy link
Contributor

A problem of removing the description is that it breaks kubectl explain

@blackpiglet
Copy link
Contributor Author

Right. kubectl explain should depend on the CRD definition YAML file. Most information is lost after this change.
Need to check whether this is acceptable for users.

@reasonerjt
Copy link
Contributor

I believe updating the pod.spec should be sufficient?

@blackpiglet
Copy link
Contributor Author

blackpiglet commented Jul 6, 2022

I believe updating the pod.spec should be sufficient?

In most cases, I think yes, but I'm not sure about Fedex NSX environment. I remember there are more than one CRDs failed during creation. Need to test to make sure.

@shubham-pampattiwar
Copy link
Collaborator

I agree with @reasonerjt, IMO breaking kubectl explain is not a good idea. Maybe try to make the descriptions more concise ? Unsure how much that can help.

@blackpiglet
Copy link
Contributor Author

I think there are two ways:

  • Make the description shorter.
  • Limit the description size in generated CRD.

Maybe we can do both.

@blackpiglet
Copy link
Contributor Author

@shubham-pampattiwar @reasonerjt
As discussion, I add the description of CRDs backup. Please take a look.

@codecov-commenter
Copy link

codecov-commenter commented Jul 12, 2022

Codecov Report

Merging #5089 (c2bbeea) into main (2778d54) will decrease coverage by 0.02%.
The diff coverage is 34.69%.

@@            Coverage Diff             @@
##             main    #5089      +/-   ##
==========================================
- Coverage   41.30%   41.27%   -0.03%     
==========================================
  Files         210      211       +1     
  Lines       18439    18478      +39     
==========================================
+ Hits         7616     7627      +11     
- Misses      10253    10276      +23     
- Partials      570      575       +5     
Impacted Files Coverage Δ
pkg/builder/container_builder.go 33.78% <0.00%> (-4.10%) ⬇️
pkg/controller/restore_controller.go 63.63% <4.76%> (-2.88%) ⬇️
internal/hook/item_hook_handler.go 88.31% <80.00%> (-1.10%) ⬇️
pkg/plugin/framework/handle_panic.go 25.00% <0.00%> (-1.67%) ⬇️
pkg/backup/item_backupper.go 78.22% <0.00%> (ø)
pkg/util/kube/predicate.go 57.14% <0.00%> (ø)
...g/controller/backup_storage_location_controller.go 64.40% <0.00%> (+5.49%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2778d54...c2bbeea. Read the comment docs.

@shubham-pampattiwar
Copy link
Collaborator

@blackpiglet Are we not going to shorten the Restore CRD spec descriptions ? Do the current changes in the PR fix the size issue ?

@blackpiglet
Copy link
Contributor Author

@shubham-pampattiwar
After discussion with @reasonerjt, we agreed that starting at reducing CRD size of restores.velero.io. If there is still issue, then try to shorten the descriptions in CRD.

@shubham-pampattiwar
Copy link
Collaborator

@shubham-pampattiwar After discussion with @reasonerjt, we agreed that starting at reducing CRD size of restores.velero.io. If there is still issue, then try to shorten the descriptions in CRD.

Ok, Thank you ! VISACK !

1. Make the Restore hook.InitConatianer server side field pruing disable.
2. Remove restore patch in update-generate-crd-code.sh.
3. Modify related testcases.
4. Add Container fields validation in Restore Init hook.

Signed-off-by: Xun Jiang <[email protected]>
@blackpiglet
Copy link
Contributor Author

blackpiglet commented Jul 29, 2022

Conflict by crd.go file. If the change is accepted, I will rebase the branch and create a new PR for merge.

@blackpiglet
Copy link
Contributor Author

Close this one, due to crds.go file conflicting with main branch.
Open #5174 to resolve conflict.

@blackpiglet blackpiglet closed this Aug 2, 2022
@blackpiglet blackpiglet deleted the reduce-crd-size branch October 15, 2022 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reduce Restore CRD's size
4 participants