Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubetest2 - Add manifest template support #10559

Merged
merged 3 commits into from
Jan 13, 2021
Merged

kubetest2 - Add manifest template support #10559

merged 3 commits into from
Jan 13, 2021

Conversation

rifelpet
Copy link
Member

This adds support for defining manifest templates used in e2e tests, allowing us to test more specific configurations not exposed through create cluster cli flags. I'm envision using this for testing ACM certificates on the API NLB, additional cilium configurations, mixed instances policies, etc.

I'm updating the test-e2e make target as I go to make the presubmit job use it but soon I'll work on how we'll actually configure and invoke kubetest2 from our variety of jobs.

Specifically how we'll access these kubetest2 binaries and other static files like the upgrade scenario script or these template files, including using the appropriate kubetest2-kops version for testing on release branches.

  1. Have the periodic jobs include extra_refs to clone the kops repo and use its Makefile, kubetest2-kops, etc. The release branch jobs could clone the appropriate kops release branch.
  2. Build a new image that contains all the appropriate dependencies. It would be rebuilt on postsubmit and have separate tags per release branch.
  3. Upload the dependencies as a part of make gcs-publish-ci. This would mean the logic to download a specific version of kops would need to live outside of kubetest2 in order to also download kubetest2.

I'm leaning towards 1 because it would be simplest and easiest. There are a few concerns about version mismatch, particularly during the updown pipeline and the various version markers. For a new merge to master, the periodic jobs would immediately use the new commit's version of kubetest2-kops but wouldn't use the kops version from that commit until its gone through the pipeline. I don't think this is a big deal though.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jan 12, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 12, 2021
@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
kubetest2 - Add manifest template support
196e678 
I'm updating the test-e2e make target as I go to make the presubmit job use it but soon I'll work on how we'll actually configure and invoke kubetest2 from our variety of jobs
@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@olemarkus
Copy link
Member

1 sounds good to me.

In this case, do you want to use kops create -f instead of replace? The former is a bit more convenient wrt keys and kubeconfig.

replace could be useful if we want to spin up a cluster, then apply a new spec and validate again. E.g if we want to test an addon upgrade/migration.

@rifelpet
Copy link
Member Author

good idea. I added the SSHCredential to the template rather than kops create secret hoping that will help.

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

1 similar comment
@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

This is (finally) ready for review 👍🏻

@olemarkus
Copy link
Member

This looks good!

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 13, 2021
@k8s-ci-robot k8s-ci-robot merged commit 8e5f717 into kubernetes:master Jan 13, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Jan 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@johngmyers johngmyers Awaiting requested review from johngmyers

@olemarkus olemarkus Awaiting requested review from olemarkus

Assignees

@olemarkus olemarkus

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.20
Development

Successfully merging this pull request may close these issues.
3 participants
@rifelpet @k8s-ci-robot @olemarkus