[Proposal]Add more E2E test cases for sealer #1966
Comments
E2E test documentSealer E2E test is inspired by kind, uses a container to emulate a node by running systemd inside the container and hosting other processes with systemd. Principle of sealer k8s in dockerSealer uses（sealer/pkg/infra/container/imagecontext/base/Dockerfile）to build a basic image: After starting the container, the sealer binary is transferred to the container through ssh, and the sealer commands such as: sealer run and sealer apply are invoked through ssh to create the k8s cluster (k8s in docker), as shown in the figure below:
How to run E2E test locallyThe code in test repository is a set of Ginkgo and Gomega based integration tests that execute commands using the sealer CLI. PrerequisitesBefore you run the tests, you'll need a sealer binary in your machine executable path and install docker. Run the TestsTo run a single test or set of tests, you'll need the Ginkgo tool installed on your machine: go install github.com/onsi/ginkgo/[email protected]To install Sealer and prepare the test environment: #build sealer source code to binary for local e2e-test in containers
git clone https://github.com/sealerio/sealer.git
cd sealer/ && make build-in-docker
cp _output/bin/sealer/linux_amd64/sealer /usr/local/bin
#prepare test environment
export REGISTRY_URL={your registry}
export REGISTRY_USERNAME={user name}
export REGISTRY_PASSWORD={password}
#default test image name: docker.io/sealerio/kubernetes:v1-22-15-sealerio-2
export IMAGE_NAME={test image name}To execute the entire test suite: cd sealer && ginkgo testYou can then use the ginkgo --focus="sealer login" testYou can then use the ginkgo -v testMore ginkgo helpful info see: ginkgo --helpHow to run E2E Tests using github actionBefore we run the CI by using github aciton, please make sure that these four variables: CI is triggered when we push a branch with name starts with release or comment How to write E2E test cases for sealerE2E tests required by sealer can be divided into those that need cluster construction and those that do not. For tests that need cluster construction, container needs to be started to simulate node (such as sealer run and sealer apply). If you don't need to build a cluster, you just need to execute it on the machine (e.g., sealer pull, sealer tag, sealer build). E2E test entrySealer all E2E test files are in the The function： Writing SpecsYou can use Ginkgo allows you to hierarchically organize the specs in your suite using container nodes. Ginkgo provides three synonymous nouns for creating container nodes: For example: var _ = Describe("sealer login", func() {
Context("login docker registry", func() {
AfterEach(func() {
registry.Logout()
})
It("with correct name and password", func() {
image.CheckLoginResult(
settings.RegistryURL,
settings.RegistryUsername,
settings.RegistryPasswd,
true)
})
It("with incorrect name and password", func() {
image.CheckLoginResult(
settings.RegistryURL,
settings.RegistryPasswd,
settings.RegistryUsername,
false)
})
It("with only name", func() {
image.CheckLoginResult(
settings.RegistryURL,
settings.RegistryUsername,
"",
false)
})
It("with only password", func() {
image.CheckLoginResult(
settings.RegistryURL,
"",
settings.RegistryPasswd,
false)
})
It("with only registryURL", func() {
image.CheckLoginResult(
settings.RegistryURL,
"",
"",
false)
})
})
})The configuration of e2e test is in More information about ginkgo and gomega can be found in the documentation:Ginkgo and Gomega. Configure github action fileThe github action file for e2e test is under the For example: name: {Test name}
on:
push:
branches: "release*"
issue_comment:
types:
- created
permissions:
statuses: write
jobs:
build:
name: test
runs-on: ubuntu-latest
if: ${{ (github.event.issue.pull_request && (github.event.comment.body == '/test all' || github.event.comment.body == '/test {name}')) || github.event_name == 'push' }}
env:
GO111MODULE: on
steps:
- name: Get PR details
if: ${{ github.event_name == 'issue_comment'}}
uses: xt0rted/pull-request-comment-branch@v1
id: comment-branch
- name: Set commit status as pending
if: ${{ github.event_name == 'issue_comment'}}
uses: myrotvorets/set-commit-status-action@master
with:
sha: ${{ steps.comment-branch.outputs.head_sha }}
token: ${{ secrets.GITHUB_TOKEN }}
status: pending
- name: Github API Request
id: request
uses: octokit/[email protected]
with:
route: ${{ github.event.issue.pull_request.url }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Get PR informations
id: pr_data
run: |
echo "repo_name=${{ fromJson(steps.request.outputs.data).head.repo.full_name }}" >> $GITHUB_STATE
echo "repo_clone_url=${{ fromJson(steps.request.outputs.data).head.repo.clone_url }}" >> $GITHUB_STATE
echo "repo_ssh_url=${{ fromJson(steps.request.outputs.data).head.repo.ssh_url }}" >> $GITHUB_STATE
- name: Check out code into the Go module directory
uses: actions/checkout@v3
with:
token: ${{ secrets.GITHUB_TOKEN }}
repository: ${{fromJson(steps.request.outputs.data).head.repo.full_name}}
ref: ${{fromJson(steps.request.outputs.data).head.ref}}
path: src/github.com/sealerio/sealer
- name: Install deps
run: |
sudo su
sudo apt-get update
sudo apt-get install -y libgpgme-dev libbtrfs-dev libdevmapper-dev
sudo mkdir /var/lib/sealer
- name: Set up Go 1.17
uses: actions/setup-go@v3
with:
go-version: 1.17
id: go
- name: Install sealer and ginkgo
shell: bash
run: |
docker run --rm -v ${PWD}:/usr/src/sealer -w /usr/src/sealer registry.cn-qingdao.aliyuncs.com/sealer-io/sealer-build:v1 make linux
export SEALER_DIR=${PWD}/_output/bin/sealer/linux_amd64
echo "$SEALER_DIR" >> $GITHUB_PATH
go install github.com/onsi/ginkgo/[email protected]
go install github.com/onsi/gomega/[email protected]
GOPATH=`go env GOPATH`
echo "$GOPATH/bin" >> $GITHUB_PATH
working-directory: src/github.com/sealerio/sealer
- name: Run **{Test name}** test
shell: bash
working-directory: src/github.com/sealerio/sealer
env:
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
REGISTRY_URL: ${{ secrets.REGISTRY_URL }}
IMAGE_NAME: ${{ secrets.IMAGE_NAME}}
ACCESSKEYID: ${{ secrets.ACCESSKEYID }}
ACCESSKEYSECRET: ${{ secrets.ACCESSKEYSECRET }}
RegionID: ${{ secrets.RegionID }}
run: | # Your main focus is here
ginkgo -v --focus="sealer {test name}" test
- name: Set final commit status
uses: myrotvorets/set-commit-status-action@master
if: always()
with:
sha: ${{ steps.comment-branch.outputs.head_sha }}
token: ${{ secrets.GITHUB_TOKEN }}
status: ${{ job.status }}More information about github action can be found in the documentation:Github action. |
Issue Description
Type: feature request
We open this issue to track and summarize cases of E2E test. Suggestions and additions to this issue are welcomed!
Describe what feature you want
Our E2E test cases are currently very limited, we need add more cases.
Additional context
E2E test task list：
sealer image:
sealer cluster:
sealer alpha:
config:
registry:
Compatibility:
The text was updated successfully, but these errors were encountered: