Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: reuse docker job from actions #752

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+29 −81
Open

ci: reuse docker job from actions #752

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 3 additions & 81 deletions .github/workflows/docker-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,89 +12,11 @@ on:
permissions:
contents: read

concurrency:
# if workflow for PR or push is already running stop it, and start new one
group: poc-storage-${{ github.ref }}
cancel-in-progress: true

jobs:
storage-push:
runs-on: ubuntu-latest
call:
permissions:
contents: read
packages: write
id-token: write
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/[email protected]

- name: Log in to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/[email protected]
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Log in to the GH Container registry
if: github.event_name != 'pull_request'
uses: docker/[email protected]
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract Docker metadata
id: meta
uses: docker/[email protected]
with:
images: |
${{ github.repository }}
ghcr.io/${{ github.repository }}

- name: Get opi-api Version
run: echo "OPI_API_VERSION=$(go list -m -f '{{.Version}}' github.com/opiproject/opi-api)" >> $GITHUB_ENV

- name: Build and push Docker image
uses: docker/[email protected]
with:
context: .
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: |
${{ steps.meta.outputs.labels }},
opi-api-version=${{ env.OPI_API_VERSION }}
cache-from: type=gha
cache-to: type=gha,mode=max

storage-ci:
runs-on: ubuntu-latest
steps:
- name: configure HUGE pages
run: |
sync
echo 1 | sudo tee /proc/sys/vm/drop_caches
sudo mkdir -p /mnt/huge
grep hugetlbfs /proc/mounts || sudo mount -t hugetlbfs nodev /mnt/huge
echo 1024 | sudo tee /proc/sys/vm/nr_hugepages
echo "Check and fail if not enough"
grep 1024 /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3
- uses: docker/setup-buildx-action@5d9862498505fcac67b9f455d6e94ec0339f7b90

- name: Start containers
run: docker-compose up --build --force-recreate --detach

- name: Run Tests
run: ./scripts/tests.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where this runs now ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!
WDYT about opiproject/actions#22

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is just duplication, since we have both success of the opi-test container and then now test.sh...
maybe we can leave just shell script and do the wait inside... WDYT ? just brainstorming...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the main difference between test.sh and godpu? In tests.sh we run some checks for nvme tcp controller by means of spdk_nvme_perf/spdk_nvme_identify.
Are they really important?
Can we add similar checks to other parts like Nvme PCIe, virtio-blk, backend api?


- name: Logs
if: always()
run: docker-compose logs

- name: Stop containers
if: always()
run: docker-compose down --volumes --remove-orphans
uses: opiproject/actions/.github/workflows/docker-publish.yml@main
secrets: inherit
26 changes: 26 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ services:
condition: service_healthy
redis:
condition: service_healthy
jaeger:
Copy link
Member

@glimchb glimchb Oct 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not must, those services can be independent... you can leave it or remove it...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to keep this as a dependency. If a jaeger service is not running, it takes too long to get a cmd executed (jaeger requests are timeouted), so if we execute docker-compose up opi-spdk-server jaeger will be brought up as well

condition: service_started
command: sh -c "/opi-spdk-bridge -grpc_port=50051 -http_port=8082 -spdk_addr=/var/tmp/spdk.sock -redis_addr=redis:6379"
healthcheck:
test: grpcurl -plaintext localhost:50051 list || exit 1
Expand All @@ -77,6 +79,8 @@ services:
image: redis:7.2.2-alpine3.18
networks:
- opi
ports:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why expose the port here ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I want to run a server without a container, I need to provide an environment as well. I can type docker-compose up redis and get redis listening on the port. The same with jaeger. But if you want, I can remove this part.

- "6379:6379"
healthcheck:
test: ["CMD", "redis-cli", "--raw", "incr", "ping"]

Expand All @@ -100,6 +104,15 @@ services:
condition: service_healthy
command: curl -qkL http://opi-spdk-server:8082/v1/inventory/1/inventory/2

opi-jaeger-test:
image: curlimages/curl:8.4.0
networks:
- opi
depends_on:
opi-spdk-client:
condition: service_completed_successfully
command: sh -c 'curl -s "http://jaeger:16686/api/traces?service=opi-spdk-bridge&lookback=20m&prettyPrint=true&limit=10" | grep operationName'

opi-spdk-client:
image: ghcr.io/opiproject/godpu:main@sha256:2b55d76814befaec963e378b1764a611f37698c70fd55bfe2ddf07f06393fa28
networks:
Expand All @@ -109,5 +122,18 @@ services:
condition: service_healthy
command: storage test --addr=opi-spdk-server:50051

opi-test:
image: docker.io/namely/grpc-cli
networks:
- opi
depends_on:
opi-spdk-client:
condition: service_completed_successfully
opi-gw-test:
condition: service_completed_successfully
opi-jaeger-test:
condition: service_completed_successfully
command: ls opi-spdk-server:50051 opi_api.storage.v1.FrontendNvmeService -l

networks:
opi: