Skip to content

Commit

Permalink
Add trigger phrases and status badge for e2e tests
Browse files Browse the repository at this point in the history 
This adds the trigger phrases for the e2e tests to the README.md and
also the build badge for the periodic job.
The two e2e test sets are described a bit more in the e2e README.md.

Signed-off-by: Lennart Jern <[email protected]>
  • Loading branch information
@lentzi90
lentzi90 committed Jan 25, 2024
1 parent d6f67a8 commit 3bbfb67
Show file tree
Hide file tree
Showing 2 changed files with 69 additions and 27 deletions.
42 changes: 26 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,18 @@
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/metal3-io/baremetal-operator/badge)](https://securityscorecards.dev/viewer/?uri=github.com/metal3-io/baremetal-operator)
[![Ubuntu daily main build status](https://jenkins.nordix.org/buildStatus/icon?job=metal3_daily_main_integration_test_ubuntu&subject=Ubuntu%20daily%20main)](https://jenkins.nordix.org/view/Metal3/job/metal3_daily_main_integration_test_ubuntu/)
[![CentOS daily main build status](https://jenkins.nordix.org/buildStatus/icon?job=metal3_daily_main_integration_test_centos&subject=CentOS%20daily%20main)](https://jenkins.nordix.org/view/Metal3/job/metal3_daily_main_integration_test_centos/)
[![BMO e2e periodic main](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-bmo-e2e-test-periodic/badge/icon?subject=BMO%20e2e%20periodic%20main)](https://jenkins.nordix.org/view/Metal3%20Periodic/job/metal3-bmo-e2e-test-periodic/)

The Bare Metal Operator implements a Kubernetes API for managing bare metal
hosts. It maintains an inventory of available hosts as instances of the
`BareMetalHost` Custom Resource Definition. The Bare Metal Operator knows how
to:

* Inspect the host’s hardware details and report them on the corresponding
- Inspect the host’s hardware details and report them on the corresponding
`BareMetalHost`. This includes information about CPUs, RAM, disks, NICs, and
more.
* Provision hosts with a desired image.
* Clean a host’s disk contents before or after provisioning.
- Provision hosts with a desired image.
- Clean a host’s disk contents before or after provisioning.

More capabilities are being added regularly. See open issues and pull requests
for more information on work in progress.
Expand All @@ -24,11 +25,11 @@ components, see the [Metal³ docs](https://github.com/metal3-io/metal3-docs).

## Resources

* [API documentation](docs/api.md)
* [Setup Development Environment](docs/dev-setup.md)
* [Configuration](docs/configuration.md)
* [Testing](docs/testing.md)
* [Publishing Images](docs/publishing-images.md)
- [API documentation](docs/api.md)
- [Setup Development Environment](docs/dev-setup.md)
- [Configuration](docs/configuration.md)
- [Testing](docs/testing.md)
- [Publishing Images](docs/publishing-images.md)

## Integration tests

Expand All @@ -39,32 +40,41 @@ Eventually these tests will be replaced with BMO's own
[e2e tests](test/e2e/README.md). BMO e2e tests are currently under active
development. Here are the tests which run on different branches in BMO:

**E2E tests:**

- **/test metal3-bmo-e2e-test-pull** runs required BMO e2e tests on your PR.
Works for main and release-0.5 branches.
- **/test metal3-bmo-e2e-test-optional-pull** runs optional BMO e2e tests on
your PR. These are under active development currently.

For more details on the e2e tests, see [test/e2e/README.md](test/e2e/README.md).

**Main branch:**

* **/test-centos-e2e-integration-main** runs CAPM3 e2e integration tests with
- **/test-centos-e2e-integration-main** runs CAPM3 e2e integration tests with
CAPM3 main branch and BMO **main** branch on Centos
* **/test-ubuntu-integration-main** runs ansible integration tests with CAPM3
- **/test-ubuntu-integration-main** runs ansible integration tests with CAPM3
main branch and BMO **main** branch on Ubuntu

**Release-0.5 branch:**

* **/test-centos-e2e-integration-release-1-6** runs CAPM3 e2e integration tests
- **/test-centos-e2e-integration-release-1-6** runs CAPM3 e2e integration tests
with CAPM3 release-1.6 branch and BMO **release-0.5** branch on Centos
* **/test-ubuntu-integration-release-1-6** runs ansible integration tests with
- **/test-ubuntu-integration-release-1-6** runs ansible integration tests with
CAPM3 release-1.6 branch and BMO **release-0.5** branch on Ubuntu

**Release-0.4 branch:**

* **/test-centos-e2e-integration-release-1-5** runs CAPM3 e2e integration tests
- **/test-centos-e2e-integration-release-1-5** runs CAPM3 e2e integration tests
with CAPM3 release-1.5 branch and BMO **release-0.4** branch on Centos
* **/test-ubuntu-integration-release-1-5** runs ansible integration tests with
- **/test-ubuntu-integration-release-1-5** runs ansible integration tests with
CAPM3 release-1.5 branch and BMO **release-0.4** branch on Ubuntu

**Release-0.3 branch:**

* **/test-centos-e2e-integration-release-1-4** runs CAPM3 e2e integration tests
- **/test-centos-e2e-integration-release-1-4** runs CAPM3 e2e integration tests
with CAPM3 release-1.4 branch and BMO **release-0.3** branch on Centos
* **/test-ubuntu-integration-release-1-4** runs ansible integration tests with
- **/test-ubuntu-integration-release-1-4** runs ansible integration tests with
CAPM3 release-1.4 branch and BMO **release-0.3** branch on Ubuntu

### Important Notes
Expand Down
54 changes: 43 additions & 11 deletions test/e2e/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ in `config`.
In the CI system we set up a pre-configured minikube cluster that is then used
for the tests, as seen in [ci-e2e.sh](../../hack/ci-e2e.sh). This allows us to
have control over the network and have a static configuration for Ironic and the
Baremetal Operator. The script creates a VM to represent a BareMetalHost,
and configures either VBMC or sushy-tools to be used as BMC.
Baremetal Operator. The script creates a VM to represent a BareMetalHost, and
configures either VBMC or sushy-tools to be used as BMC.

Currently there are two sets of tests, which cannot be ran together in the same
cluster. One is the "optional" set, currently consists of only the [upgrade tests](upgrade_test.go),
and the "main" set, which are the ones required to run in every BMO PR. One can
switch between these sets by manipulating the `GINKGO_FOCUS` and `GINKGO_SKIP` env
vars. In the default setting, the script sets `GINKGO_SKIP` to `upgrade`.
cluster. One is the "optional" set, currently consists of only the
[upgrade tests](upgrade_test.go), and the "main" set, which are the ones
required to run in every BMO PR. One can switch between these sets by
manipulating the `GINKGO_FOCUS` and `GINKGO_SKIP` env vars. In the default
setting, the script sets `GINKGO_SKIP` to `upgrade`.

E.g. Here is how to run the E2E main tests:

Expand All @@ -34,13 +35,44 @@ export GINKGO_FOCUS="upgrade"
```

It is also possible to run the tests with the fixture provider instead of
Ironic. Without any changes, the whole suite (including optional tests) will be run.
Please note, however, that it is quite questionable to call this configuration
"end to end".
Ironic. Without any changes, the whole suite (including optional tests) will be
run. Please note, however, that it is quite questionable to call this
configuration "end to end".

Fixture provider is configured to be the default one. This is how to run
the e2e test suite with it:
Fixture provider is configured to be the default one. This is how to run the e2e
test suite with it:

```bash
make test-e2e
```

## Tests

Here is a list of the currently implemented tests. As mentioned above, we divide
them into two sets, one required and one optional. The required tests must pass
on every PR before merging. Optional tests may be triggered for extra
verification but are not expected to run on all PRs. All tests are checked with
both redfish and ipmi protocols in CI.

**Required tests:**

- Inspection: Check that a BareMetalHost is registered and inspected when
created, and that it becomes available after the inspection.
- External inspection: Check that a BareMetalHost, with inspection disabled and
hardware details added through an annotation, skips inspection, accepts the
hardware details from the annotation and becomes available.
- Re-inspection: Check that an available BareMetalHost is re-inspected when the
inspect annotation is added and that this updates the hardware details.
- Provisioning: Check that an available BareMetalHost can be provisioned, that
it can be detatched, deleted and re-created, with the status annotation,
without affecting the host. Finally checks that the BareMetalHost can be
deprovisioned and becomes available again.
- Live-ISO: Check that an available BareMetalHost can be provisioned with a
live-ISO image.

**Optional tests:**

- Bare Metal Operator upgrade: Check that an older version of the Bare Metal
Operator works by creating a BareMetalHost with external inspection. Then
upgrade the Bare Metal Operator and check the the BareMetalHost can be
provisioned with the upgraded version.

0 comments on commit 3bbfb67

Please sign in to comment.