Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration tests for DiscordApp #723

Merged
merged 22 commits into from
Sep 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
5f8f944
Added Discord e2e tests while minding differences with existing slack…
ezodude Sep 6, 2022
6276993
Updated YAML config with required fields.
ezodude Sep 6, 2022
7a904b2
First stab at unifying Slack and Discord interfaces.
ezodude Sep 7, 2022
872410a
Unified channel initialisation for Slack and Discord.
ezodude Sep 7, 2022
c45bc66
Slack and Discord drivers can interchangeably run the same tests.
ezodude Sep 7, 2022
37efaac
E2E test rework is complete.
ezodude Sep 8, 2022
6cf06af
Linter fixes.
ezodude Sep 8, 2022
e6a031c
TesterName use botkube_tester by default.
ezodude Sep 8, 2022
0382af0
Setup e2e tests to run with helm test and CI.
ezodude Sep 9, 2022
07aa0b2
Updated chart readme to include new e2e discord settings.
ezodude Sep 9, 2022
7cf5e96
Updated E2E tests documentation.
ezodude Sep 9, 2022
304c022
Ensured branch-build and pr-build GitHub actions are configured corre…
ezodude Sep 15, 2022
a4d6e69
Merged in interactive help from main.
ezodude Sep 15, 2022
1d1a9f5
Remove debug log level setting to avoid noisy e2e tests.
ezodude Sep 16, 2022
fbf3524
Driver based E2E tests now support interactive message checks.
ezodude Sep 16, 2022
76550b0
Merge branch 'main' into e2e-discord
ezodude Sep 16, 2022
d11c508
Updated Slack test message wait time out.
ezodude Sep 16, 2022
dbca94f
Updated e2e tests README per review comments.
ezodude Sep 16, 2022
14fc98f
Added missing compiler directives.
ezodude Sep 16, 2022
3ecda16
Removed legacy tests.
ezodude Sep 16, 2022
e903647
Linter fixes.
ezodude Sep 16, 2022
7b2378f
Added missing registry setting when building single e2e container.
ezodude Sep 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .github/workflows/branch-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
env:
HELM_VERSION: v3.9.0
K3D_VERSION: v5.4.3
INTEGRATION_TESTS_TIMEOUT: 10m
INTEGRATION_TESTS_TIMEOUT: 30m
IMAGE_REGISTRY: "ghcr.io"
IMAGE_REPOSITORY: "kubeshop/botkube"
TEST_IMAGE_REPOSITORY: "kubeshop/botkube-test"
Expand Down Expand Up @@ -82,7 +82,11 @@ jobs:
- name: Install BotKube
env:
SLACK_BOT_TOKEN: ${{ secrets.SOCKET_SLACK_BOT_TOKEN }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
DISCORD_BOT_ID: ${{ secrets.DISCORD_BOT_ID }}
SLACK_TESTER_APP_TOKEN: ${{ secrets.SLACK_TESTER_APP_TOKEN }}
DISCORD_TESTER_APP_TOKEN: ${{ secrets.DISCORD_TESTER_APP_TOKEN }}
DISCORD_GUILD_ID: ${{ secrets.DISCORD_GUILD_ID }}
run: |
helm install botkube --namespace botkube ./helm/botkube --wait --create-namespace \
-f ./helm/botkube/e2e-test-values.yaml \
Expand All @@ -94,8 +98,11 @@ jobs:
--set e2eTest.image.repository="${TEST_IMAGE_REPOSITORY}" \
--set e2eTest.image.tag="${IMAGE_TAG}" \
--set e2eTest.slack.testerAppToken="${SLACK_TESTER_APP_TOKEN}" \
--set e2eTest.slack.botName="botkubev2" \
--set e2eTest.slack.additionalContextMessage="Branch test - commit SHA: ${GITHUB_SHA} - https://github.com/kubeshop/botkube/commit/${GITHUB_SHA}" \
--set e2eTest.slack.botName="botkubev2"
--set e2eTest.discord.testerAppToken="${DISCORD_TESTER_APP_TOKEN}" \
--set e2eTest.discord.guildID="${DISCORD_GUILD_ID}" \
--set e2eTest.discord.additionalContextMessage="Branch test - commit SHA: ${GITHUB_SHA} - https://github.com/kubeshop/botkube/commit/${GITHUB_SHA}"

- name: Run tests
run: "helm test botkube --namespace botkube --timeout=$INTEGRATION_TESTS_TIMEOUT --logs"
11 changes: 9 additions & 2 deletions .github/workflows/pr-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ env:
HELM_VERSION: v3.9.0
K3D_VERSION: v5.4.3
PR_NUMBER: ${{ github.event.pull_request.number }}
INTEGRATION_TESTS_TIMEOUT: 10m
INTEGRATION_TESTS_TIMEOUT: 30m
IMAGE_REGISTRY: "ghcr.io"
IMAGE_REPOSITORY: "kubeshop/pr/botkube"
TEST_IMAGE_REPOSITORY: "kubeshop/pr/botkube-test"
Expand Down Expand Up @@ -157,7 +157,11 @@ jobs:
- name: Install BotKube
env:
SLACK_BOT_TOKEN: ${{ secrets.SOCKET_SLACK_BOT_TOKEN }}
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
DISCORD_BOT_ID: ${{ secrets.DISCORD_BOT_ID }}
SLACK_TESTER_APP_TOKEN: ${{ secrets.SLACK_TESTER_APP_TOKEN }}
DISCORD_TESTER_APP_TOKEN: ${{ secrets.DISCORD_TESTER_APP_TOKEN }}
DISCORD_GUILD_ID: ${{ secrets.DISCORD_GUILD_ID }}
run: |
helm install botkube --namespace botkube ./helm/botkube --wait --create-namespace \
-f ./helm/botkube/e2e-test-values.yaml \
Expand All @@ -170,7 +174,10 @@ jobs:
--set e2eTest.image.tag="${IMAGE_TAG}" \
--set e2eTest.slack.testerAppToken="${SLACK_TESTER_APP_TOKEN}" \
--set e2eTest.slack.additionalContextMessage="Pull request: ${PR_NUMBER} - https://github.com/kubeshop/botkube/pull/${PR_NUMBER}" \
--set e2eTest.slack.botName="botkubev2"
--set e2eTest.slack.botName="botkubev2" \
--set e2eTest.discord.testerAppToken="${DISCORD_TESTER_APP_TOKEN}" \
--set e2eTest.discord.guildID="${DISCORD_GUILD_ID}" \
--set e2eTest.discord.additionalContextMessage="Pull request: ${PR_NUMBER} - https://github.com/kubeshop/botkube/pull/${PR_NUMBER}"

- name: Run tests
run: "helm test botkube --namespace botkube --timeout=$INTEGRATION_TESTS_TIMEOUT --logs"
17 changes: 10 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
TAG=$(shell cut -d'=' -f2- .release)

.DEFAULT_GOAL := build
.PHONY: release git-tag check-git-status container-image test test-integration build pre-build publish lint lint-fix go-import-fmt system-check save-images load-and-push-images
.PHONY: release git-tag check-git-status container-image test test-integration-slack test-integration-discord build pre-build publish lint lint-fix go-import-fmt system-check save-images load-and-push-images

# Show this help.
help:
Expand Down Expand Up @@ -40,8 +40,11 @@ go-import-fmt:
test: system-check
@go test -v -race ./...

test-integration: system-check
@go test -v -tags=integration -race -count=1 ./test/...
test-integration-slack: system-check
@go test -v -tags=integration -race -count=1 ./test/... -run "TestSlack"

test-integration-discord: system-check
@go test -v -tags=integration -race -count=1 ./test/... -run "TestDiscord"

# Build the binary
build: pre-build
Expand All @@ -60,11 +63,11 @@ container-image-single: pre-build
@./hack/goreleaser.sh build_single
@echo "Single target docker image build successful"

# Build the test image
container-image-test-single: pre-build
# Build the e2e test image
container-image-single-e2e: pre-build
@echo "Building single target docker image for e2e test"
@./hack/goreleaser.sh build_test_single
@echo "Single target docker image build for e2e test successful"
@./hack/goreleaser.sh build_single_e2e
@echo "Single target docker image build for e2e tests successful"

# Publish release using goreleaser
gorelease:
Expand Down
10 changes: 5 additions & 5 deletions hack/goreleaser.sh
Original file line number Diff line number Diff line change
Expand Up @@ -114,16 +114,16 @@ build_single() {
rm "$PWD/botkube"
}

build_test_single() {
build_single_e2e(){
export GORELEASER_CURRENT_TAG=v9.99.9-dev
docker run --rm --privileged \
-v "$PWD":/go/src/github.com/kubeshop/botkube \
-v /var/run/docker.sock:/var/run/docker.sock \
-w /go/src/github.com/kubeshop/botkube \
-e GORELEASER_CURRENT_TAG=${GORELEASER_CURRENT_TAG} \
goreleaser/goreleaser build --single-target --rm-dist --snapshot --id botkube-test -o "./botkube-e2e.test"
docker build -f "$PWD/build/test.Dockerfile" --platform "${IMAGE_PLATFORM}" -t "${IMAGE_REGISTRY}/${TEST_IMAGE_REPOSITORY}:${GORELEASER_CURRENT_TAG}" --build-arg TEST_NAME=botkube-e2e.test .
rm "$PWD/botkube-test"
docker build -f "$PWD/build/test.Dockerfile" --build-arg=TEST_NAME=botkube-e2e.test --platform "${IMAGE_PLATFORM}" -t "${IMAGE_REGISTRY}/${TEST_IMAGE_REPOSITORY}:${GORELEASER_CURRENT_TAG}" .
rm "$PWD/botkube-e2e.test"
}

release() {
Expand Down Expand Up @@ -156,8 +156,8 @@ case "${1}" in
build_single)
build_single
;;
build_test_single)
build_test_single
build_single_e2e)
build_single_e2e
;;
release)
release
Expand Down
6 changes: 6 additions & 0 deletions helm/botkube/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,12 @@ Controller for the BotKube Slack app which helps you monitor your Kubernetes clu
| [e2eTest.slack.testerAppToken](./values.yaml#L645) | string | `""` | Slack tester application token that interacts with BotKube bot. |
| [e2eTest.slack.additionalContextMessage](./values.yaml#L647) | string | `""` | Additional message that is sent by Tester. You can pass e.g. pull request number or source link where these tests are run from. |
| [e2eTest.slack.messageWaitTimeout](./values.yaml#L649) | string | `"1m"` | Message wait timeout. It defines how long we wait to ensure that notification were not sent when disabled. |
| [e2eTest.discord.botName](./values.yaml#L652) | string | `"botkube"` | Name of the BotKube bot to interact with during the e2e tests. |
| [e2eTest.discord.testerName](./values.yaml#L654) | string | `"botkube_tester"` | Name of the BotKube Tester bot that sends messages during the e2e tests. |
| [e2eTest.discord.guildID](./values.yaml#L656) | string | `""` | Discord Guild ID (discord server ID) used to run e2e tests |
| [e2eTest.discord.testerAppToken](./values.yaml#L658) | string | `""` | Discord tester application token that interacts with BotKube bot. |
| [e2eTest.discord.additionalContextMessage](./values.yaml#L660) | string | `""` | Additional message that is sent by Tester. You can pass e.g. pull request number or source link where these tests are run from. |
| [e2eTest.discord.messageWaitTimeout](./values.yaml#L662) | string | `"1m"` | Message wait timeout. It defines how long we wait to ensure that notification were not sent when disabled. |

### AWS IRSA on EKS support

Expand Down
34 changes: 31 additions & 3 deletions helm/botkube/e2e-test-values.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
analytics:
disable: true

communications:
'default-group':
slack:
Expand All @@ -21,6 +24,30 @@ communications:
- kubectl-read-only
sources:
- k8s-updates
discord:
enabled: false # Tests will override this temporarily
token: "" # Provide a valid token for BotKube app
botID: "" # Provide a valid Application Client ID for BotKube app
huseyinbabal marked this conversation as resolved.
Show resolved Hide resolved
channels:
'default':
id: "" # Tests will override this channel ID temporarily
bindings:
executors:
- kubectl-read-only
- kubectl-wait-cmd
- kubectl-exec-cmd
- kubectl-allow-all
sources:
- k8s-events
'secondary':
id: "" # Tests will override this channel ID temporarily
bindings:
# -- Executors configuration for a given channel.
executors:
- kubectl-read-only
# -- Notification sources configuration for a given channel.
sources:
- k8s-updates

sources:
'k8s-events':
Expand Down Expand Up @@ -107,10 +134,11 @@ settings:
extraAnnotations:
botkube.io/disable: "true"

analytics:
ezodude marked this conversation as resolved.
Show resolved Hide resolved
disable: true

e2eTest:
slack:
testerAppToken: "" # Provide a valid token for BotKube tester app
additionalContextMessage: "" # Optional additional context
discord:
guildID: "" # Provide the Guild ID (discord server ID) used to run e2e tests
testerAppToken: "" # Provide a valid token for BotKube tester app
additionalContextMessage: "" # Optional additional context
20 changes: 18 additions & 2 deletions helm/botkube/templates/tests/e2e-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@ spec:
env:
- name: CONFIG_PATH
value: "/config/"
- name: ADDITIONAL_CONTEXT_MESSAGE
value: "{{ .Values.e2eTest.additionalContextMessage }}"
- name: DEPLOYMENT_NAME
value: "{{ include "botkube.fullname" . }}"
- name: DEPLOYMENT_NAMESPACE
Expand All @@ -36,16 +34,34 @@ spec:
value: "BOTKUBE_COMMUNICATIONS_DEFAULT-GROUP_SLACK_CHANNELS_DEFAULT_NAME"
- name: DEPLOYMENT_ENVS_SECONDARY_SLACK_CHANNEL_ID_NAME
value: "BOTKUBE_COMMUNICATIONS_DEFAULT-GROUP_SLACK_CHANNELS_SECONDARY_NAME"
- name: DEPLOYMENT_ENVS_DISCORD_ENABLED_NAME
value: "BOTKUBE_COMMUNICATIONS_DEFAULT-GROUP_DISCORD_ENABLED"
- name: DEPLOYMENT_ENVS_DEFAULT_DISCORD_CHANNEL_ID_NAME
value: "BOTKUBE_COMMUNICATIONS_DEFAULT-GROUP_DISCORD_CHANNELS_DEFAULT_ID"
- name: DEPLOYMENT_ENVS_SECONDARY_DISCORD_CHANNEL_ID_NAME
value: "BOTKUBE_COMMUNICATIONS_DEFAULT-GROUP_DISCORD_CHANNELS_SECONDARY_ID"
- name: CLUSTER_NAME
value: "{{ .Values.settings.clusterName }}"
- name: SLACK_BOT_NAME
value: "{{ .Values.e2eTest.slack.botName }}"
- name: DISCORD_BOT_NAME
value: "{{ .Values.e2eTest.discord.botName }}"
- name: SLACK_TESTER_NAME
value: "{{ .Values.e2eTest.slack.testerName }}"
- name: DISCORD_TESTER_NAME
value: "{{ .Values.e2eTest.discord.testerName }}"
- name: SLACK_ADDITIONAL_CONTEXT_MESSAGE
value: "{{ .Values.e2eTest.slack.additionalContextMessage }}"
- name: DISCORD_ADDITIONAL_CONTEXT_MESSAGE
value: "{{ .Values.e2eTest.discord.additionalContextMessage }}"
- name: SLACK_TESTER_APP_TOKEN
value: "{{ .Values.e2eTest.slack.testerAppToken }}"
- name: DISCORD_TESTER_APP_TOKEN
value: "{{ .Values.e2eTest.discord.testerAppToken }}"
- name: DISCORD_GUILD_ID
value: "{{ .Values.e2eTest.discord.guildID }}"
- name: SLACK_MESSAGE_WAIT_TIMEOUT
value: "{{ .Values.e2eTest.slack.messageWaitTimeout }}"
- name: DISCORD_MESSAGE_WAIT_TIMEOUT
value: "{{ .Values.e2eTest.discord.messageWaitTimeout }}"
restartPolicy: Never
13 changes: 13 additions & 0 deletions helm/botkube/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -647,3 +647,16 @@ e2eTest:
additionalContextMessage: ""
# -- Message wait timeout. It defines how long we wait to ensure that notification were not sent when disabled.
messageWaitTimeout: 1m
discord:
# -- Name of the BotKube bot to interact with during the e2e tests.
botName: "botkube"
# -- Name of the BotKube Tester bot that sends messages during the e2e tests.
testerName: "botkube_tester"
# -- Discord Guild ID (discord server ID) used to run e2e tests
guildID: ""
# -- Discord tester application token that interacts with BotKube bot.
testerAppToken: ""
# -- Additional message that is sent by Tester. You can pass e.g. pull request number or source link where these tests are run from.
additionalContextMessage: ""
# -- Message wait timeout. It defines how long we wait to ensure that notification were not sent when disabled.
messageWaitTimeout: 1m
Loading