diff --git a/README.md b/README.md index 3af99f04a8..1a922b20d4 100644 --- a/README.md +++ b/README.md @@ -4,17 +4,18 @@ [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/metal3-io/baremetal-operator/badge)](https://securityscorecards.dev/viewer/?uri=github.com/metal3-io/baremetal-operator) [![Ubuntu daily main build status](https://jenkins.nordix.org/buildStatus/icon?job=metal3_daily_main_integration_test_ubuntu&subject=Ubuntu%20daily%20main)](https://jenkins.nordix.org/view/Metal3/job/metal3_daily_main_integration_test_ubuntu/) [![CentOS daily main build status](https://jenkins.nordix.org/buildStatus/icon?job=metal3_daily_main_integration_test_centos&subject=CentOS%20daily%20main)](https://jenkins.nordix.org/view/Metal3/job/metal3_daily_main_integration_test_centos/) +[![BMO e2e periodic main](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-bmo-e2e-test-periodic/badge/icon?subject=BMO%20e2e%20periodic%20main)](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-bmo-e2e-test-periodic/) The Bare Metal Operator implements a Kubernetes API for managing bare metal hosts. It maintains an inventory of available hosts as instances of the `BareMetalHost` Custom Resource Definition. The Bare Metal Operator knows how to: -* Inspect the host’s hardware details and report them on the corresponding +- Inspect the host’s hardware details and report them on the corresponding `BareMetalHost`. This includes information about CPUs, RAM, disks, NICs, and more. -* Provision hosts with a desired image. -* Clean a host’s disk contents before or after provisioning. +- Provision hosts with a desired image. +- Clean a host’s disk contents before or after provisioning. More capabilities are being added regularly. See open issues and pull requests for more information on work in progress. @@ -24,11 +25,11 @@ components, see the [Metal³ docs](https://github.com/metal3-io/metal3-docs). ## Resources -* [API documentation](docs/api.md) -* [Setup Development Environment](docs/dev-setup.md) -* [Configuration](docs/configuration.md) -* [Testing](docs/testing.md) -* [Publishing Images](docs/publishing-images.md) +- [API documentation](docs/api.md) +- [Setup Development Environment](docs/dev-setup.md) +- [Configuration](docs/configuration.md) +- [Testing](docs/testing.md) +- [Publishing Images](docs/publishing-images.md) ## Integration tests @@ -39,32 +40,41 @@ Eventually these tests will be replaced with BMO's own [e2e tests](test/e2e/README.md). BMO e2e tests are currently under active development. Here are the tests which run on different branches in BMO: +**E2E tests:** + +- **/test metal3-bmo-e2e-test-pull** runs required BMO e2e tests on your PR. + Works for main and release-0.5 branches. +- **/test metal3-bmo-e2e-test-optional-pull** runs optional BMO e2e tests on + your PR. These are under active development currently. + +For more details on the e2e tests, see [test/e2e/README.md](test/e2e/README.md). + **Main branch:** -* **/test-centos-e2e-integration-main** runs CAPM3 e2e integration tests with +- **/test-centos-e2e-integration-main** runs CAPM3 e2e integration tests with CAPM3 main branch and BMO **main** branch on Centos -* **/test-ubuntu-integration-main** runs ansible integration tests with CAPM3 +- **/test-ubuntu-integration-main** runs ansible integration tests with CAPM3 main branch and BMO **main** branch on Ubuntu **Release-0.5 branch:** -* **/test-centos-e2e-integration-release-1-6** runs CAPM3 e2e integration tests +- **/test-centos-e2e-integration-release-1-6** runs CAPM3 e2e integration tests with CAPM3 release-1.6 branch and BMO **release-0.5** branch on Centos -* **/test-ubuntu-integration-release-1-6** runs ansible integration tests with +- **/test-ubuntu-integration-release-1-6** runs ansible integration tests with CAPM3 release-1.6 branch and BMO **release-0.5** branch on Ubuntu **Release-0.4 branch:** -* **/test-centos-e2e-integration-release-1-5** runs CAPM3 e2e integration tests +- **/test-centos-e2e-integration-release-1-5** runs CAPM3 e2e integration tests with CAPM3 release-1.5 branch and BMO **release-0.4** branch on Centos -* **/test-ubuntu-integration-release-1-5** runs ansible integration tests with +- **/test-ubuntu-integration-release-1-5** runs ansible integration tests with CAPM3 release-1.5 branch and BMO **release-0.4** branch on Ubuntu **Release-0.3 branch:** -* **/test-centos-e2e-integration-release-1-4** runs CAPM3 e2e integration tests +- **/test-centos-e2e-integration-release-1-4** runs CAPM3 e2e integration tests with CAPM3 release-1.4 branch and BMO **release-0.3** branch on Centos -* **/test-ubuntu-integration-release-1-4** runs ansible integration tests with +- **/test-ubuntu-integration-release-1-4** runs ansible integration tests with CAPM3 release-1.4 branch and BMO **release-0.3** branch on Ubuntu ### Important Notes diff --git a/test/e2e/README.md b/test/e2e/README.md index fb470749b5..b77dea668b 100644 --- a/test/e2e/README.md +++ b/test/e2e/README.md @@ -11,14 +11,15 @@ in `config`. In the CI system we set up a pre-configured minikube cluster that is then used for the tests, as seen in [ci-e2e.sh](../../hack/ci-e2e.sh). This allows us to have control over the network and have a static configuration for Ironic and the -Baremetal Operator. The script creates a VM to represent a BareMetalHost, -and configures either VBMC or sushy-tools to be used as BMC. +Baremetal Operator. The script creates a VM to represent a BareMetalHost, and +configures either VBMC or sushy-tools to be used as BMC. Currently there are two sets of tests, which cannot be ran together in the same -cluster. One is the "optional" set, currently consists of only the [upgrade tests](upgrade_test.go), -and the "main" set, which are the ones required to run in every BMO PR. One can -switch between these sets by manipulating the `GINKGO_FOCUS` and `GINKGO_SKIP` env -vars. In the default setting, the script sets `GINKGO_SKIP` to `upgrade`. +cluster. One is the "optional" set, currently consists of only the +[upgrade tests](upgrade_test.go), and the "main" set, which are the ones +required to run in every BMO PR. One can switch between these sets by +manipulating the `GINKGO_FOCUS` and `GINKGO_SKIP` env vars. In the default +setting, the script sets `GINKGO_SKIP` to `upgrade`. E.g. Here is how to run the E2E main tests: @@ -34,13 +35,44 @@ export GINKGO_FOCUS="upgrade" ``` It is also possible to run the tests with the fixture provider instead of -Ironic. Without any changes, the whole suite (including optional tests) will be run. -Please note, however, that it is quite questionable to call this configuration -"end to end". +Ironic. Without any changes, the whole suite (including optional tests) will be +run. Please note, however, that it is quite questionable to call this +configuration "end to end". -Fixture provider is configured to be the default one. This is how to run -the e2e test suite with it: +Fixture provider is configured to be the default one. This is how to run the e2e +test suite with it: ```bash make test-e2e ``` + +## Tests + +Here is a list of the currently implemented tests. As mentioned above, we divide +them into two sets, one required and one optional. The required tests must pass +on every PR before merging. Optional tests may be triggered for extra +verification but are not expected to run on all PRs. All tests are checked with +both redfish and ipmi protocols in CI. + +**Required tests:** + +- Inspection: Check that a BareMetalHost is registered and inspected when + created, and that it becomes available after the inspection. +- External inspection: Check that a BareMetalHost, with inspection disabled and + hardware details added through an annotation, skips inspection, accepts the + hardware details from the annotation and becomes available. +- Re-inspection: Check that an available BareMetalHost is re-inspected when the + inspect annotation is added and that this updates the hardware details. +- Provisioning: Check that an available BareMetalHost can be provisioned, that + it can be detatched, deleted and re-created, with the status annotation, + without affecting the host. Finally checks that the BareMetalHost can be + deprovisioned and becomes available again. +- Live-ISO: Check that an available BareMetalHost can be provisioned with a + live-ISO image. + +**Optional tests:** + +- Bare Metal Operator upgrade: Check that an older version of the Bare Metal + Operator works by creating a BareMetalHost with external inspection. Then + upgrade the Bare Metal Operator and check the the BareMetalHost can be + provisioned with the upgraded version.