Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testsys: Add support for metal-k8s testing #2871

Merged
merged 1 commit into from
Apr 17, 2023

Conversation

ecpullen
Copy link
Contributor

@ecpullen ecpullen commented Mar 8, 2023

Issue number:

Closes #2152

Description of changes:

Adds support for testing metal k8s variants with testsys. There are several requirements for testing with metal that will be described in TESTING.md.

Testing done:

Ran cargo make -e BUILDSYS_VARIANT="aws-k8s-1.24" test with sample files and made sure the crd was properly created.

cargo make -e BUILDSYS_VARIANT="metal-k8s-1.24"
# Convert lz4 to gzip and push
cargo make -e BUILDSYS_VARIANT="metal-k8s-1.24" -e TESTSYS_MGMT_CLUSTER_KUBECONFIG=tests/metal-mgmt.kubeconfig test

Test passed after the cluster finished provisioning.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

@@ -261,6 +261,14 @@ pub struct GenericVariantConfig {
pub control_plane_endpoint: Option<String>,
/// The path to userdata that should be used for Bottlerocket launch
pub userdata: Option<String>,
/// The directory containing Bottlerocket images. For metal, this is the directory containing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's pretty awkward the way this config has things that are required for some variants and other things that are required for other variants. I don't suppose there's much we can do about it right now, but it's worth thinking about. There are different configuration "schemas" for different provisioning scenarios.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 2 things that need to be considered.

  • It is nice that we can set a value for aws and have it work for aws-k8s and aws-ecs variants.
  • The reduced_config fn works really well and simply since all GenericConfigs are the same.

tools/testsys-config/src/lib.rs Show resolved Hide resolved
tools/testsys-config/src/lib.rs Outdated Show resolved Hide resolved
tools/testsys/Cargo.toml Outdated Show resolved Hide resolved
tools/testsys/src/crds.rs Show resolved Hide resolved
@ecpullen ecpullen force-pushed the testsys-metal branch 3 times, most recently from 0004592 to 5ea1938 Compare March 9, 2023 19:36
@ecpullen ecpullen marked this pull request as ready for review March 13, 2023 20:33
@ecpullen ecpullen requested a review from webern March 14, 2023 16:38
TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
@@ -65,6 +66,10 @@ pub(crate) struct Run {
#[clap(long, env = "BUILDSYS_OVA")]
ova_name: Option<String>,

/// The name of the image that should be used for Bare Metal testing
#[clap(long, env = "BUILDSYS_NAME_FULL")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this include the extension? i.e. is this the file name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a cargo make env variable that we are re-using.

Copy link
Member

@gthao313 gthao313 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
TESTING.md Outdated Show resolved Hide resolved
In `Test.toml` set `cluster-names = ["cluster1"]` to tell TestSys that we want the cluster config and hardware csv from the directory we just created.

Metal testing also requires and additional manual step for testing.
The Bottlerocket build system compresses the metal images with lz4, but EKS Anywhere requires them to be gzipped, so before testing make sure to uncompress the lz4 image and gzip it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side-note: it would be nice to have a cargo-make task that does this for us.

tools/testsys/src/metal_k8s.rs Outdated Show resolved Hide resolved
tools/testsys/src/metal_k8s.rs Outdated Show resolved Hide resolved
@ecpullen ecpullen merged commit 20e81f1 into bottlerocket-os:develop Apr 17, 2023
@ecpullen ecpullen deleted the testsys-metal branch April 17, 2023 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cargo make test for metal-k8s variants
4 participants