Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: potential CI injections #826

Merged
merged 4 commits into from
Jan 29, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/pull-request-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-lint
uses: ./actions/ci-lint-ts
Expand All @@ -25,6 +27,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-prettier
uses: ./actions/ci-prettier
Expand All @@ -38,6 +42,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-lint-misc
uses: ./actions/ci-lint-misc
Expand All @@ -58,6 +64,7 @@ jobs:
# Commit back any changes based on the commit that triggered this action
# rather than merge commit of main into the PR branch
ref: ${{ github.event.pull_request.head.sha }}
persist-credentials: false

- name: Setup pnpm
uses: ./actions/setup-nodejs
Expand All @@ -81,6 +88,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: Setup pnpm
uses: ./actions/setup-nodejs
Expand All @@ -95,6 +104,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-test
uses: ./actions/ci-test-ts
Expand All @@ -108,6 +119,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-build-artifacts
uses: ./actions/cicd-build-publish-artifacts-ts
Expand All @@ -121,6 +134,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: Setup pnpm
uses: ./actions/setup-nodejs
Expand All @@ -140,6 +155,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: Setup pnpm
uses: ./actions/setup-nodejs
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/push-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-prettier
uses: ./actions/ci-prettier
Expand All @@ -37,6 +39,8 @@ jobs:
steps:
- name: Checkout repo (needed to reference local action)
uses: actions/[email protected]
with:
persist-credentials: false

- name: ci-test
uses: ./actions/ci-test-ts
Expand All @@ -59,6 +63,7 @@ jobs:
uses: actions/[email protected]
with:
fetch-depth: 0
persist-credentials: false

- name: cicd-build-publish-artifacts-dev
uses: ./actions/cicd-build-publish-artifacts-ts
Expand All @@ -77,6 +82,7 @@ jobs:
uses: actions/[email protected]
with:
fetch-depth: 0
persist-credentials: false

- name: cd-release
uses: ./actions/cicd-changesets
Expand Down
82 changes: 64 additions & 18 deletions .github/workflows/run-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -299,23 +299,27 @@ jobs:
steps:
- name: Check input conditions
id: check-inputs
env:
TEST_IDS: ${{ inputs.test_ids }}
TEST_TRIGGER: ${{ inputs.test_trigger }}
TEST_SECRETS_OVERRIDE_KEY: ${{ inputs.test_secrets_override_key }}
run: |
if [[ "${{ inputs.test_ids }}" != "" && "${{ inputs.test_trigger }}" != "" ]]; then
if [[ "$TEST_IDS" != "" && "$TEST_TRIGGER" != "" ]]; then
echo "::error::Error: Both 'test_ids' and 'test_trigger' are provided. Please specify only one."
exit 1
fi

# Check if both TEST_SECRETS_OVERRIDE_BASE64 and test_secrets_override_key starting with 'aws:' are set
if [[ "${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }}" != "" && "${{ inputs.test_secrets_override_key }}" == aws:* ]]; then
if [[ "${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }}" != "" && "$TEST_SECRETS_OVERRIDE_KEY" == aws:* ]]; then
echo "::error::Error: Both GitHub Secret and AWS Secret ('test_secrets_override_key' starting with 'aws:') are set. Please specify only one."
exit 1
fi

# Inform if custom secrets are being used
if [[ "${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }}" != "" ]]; then
echo "Will run tests with custom test secrets from GitHub Secret."
elif [[ "${{ inputs.test_secrets_override_key }}" == aws:* ]]; then
ORIGINAL_KEY="${{ inputs.test_secrets_override_key }}"
elif [[ "$TEST_SECRETS_OVERRIDE_KEY" == aws:* ]]; then
ORIGINAL_KEY="$TEST_SECRETS_OVERRIDE_KEY"
SECRET_ID="${ORIGINAL_KEY#aws:}"
echo "aws_test_secrets_key=$SECRET_ID" >> "$GITHUB_OUTPUT"
echo "Will run tests with custom test secrets from AWS Secrets Manager. AWS Secret ID: $SECRET_ID"
Expand All @@ -327,8 +331,11 @@ jobs:
- name: Create matrix for required Chainlink image versions
id: set-required-chainlink-image-versions-matrix
shell: bash
env:
REQUIRE_CHAINLINK_IMAGE_VERSIONS_IN_QA_ECR:
${{ inputs.require_chainlink_image_versions_in_qa_ecr }}
run: |
image_versions="${{ inputs.require_chainlink_image_versions_in_qa_ecr }}"
image_versions="$REQUIRE_CHAINLINK_IMAGE_VERSIONS_IN_QA_ECR"
default_version="${{ env.DEFAULT_CHAINLINK_VERSION }}"
current_sha="${{ github.sha }}"

Expand All @@ -350,8 +357,11 @@ jobs:
- name: Create matrix for required Chainlink plugin versions
id: set-required-chainlink-plugin-versions-matrix
shell: bash
env:
REQUIRE_CHAINLINK_IMAGE_VERSIONS_IN_QA_ECR:
${{ inputs.require_chainlink_image_versions_in_qa_ecr }}
run: |
image_versions=$(echo "${{ inputs.require_chainlink_plugin_versions_in_qa_ecr }}" | jq -Rc 'if . == "" then "" else split(",") | if . == [""] then "" else . end end')
image_versions=$(echo "$REQUIRE_CHAINLINK_IMAGE_VERSIONS_IN_QA_ECR" | jq -Rc 'if . == "" then "" else split(",") | if . == [""] then "" else . end end')
echo "Required Chainlink plugin image versions: $image_versions"
echo "versions=$image_versions" >> "$GITHUB_OUTPUT"

Expand All @@ -363,14 +373,19 @@ jobs:
steps:
- name: Checkout code
uses: actions/[email protected]
with:
persist-credentials: false
- name: Install citool
shell: bash
run: go install
github.com/smartcontractkit/chainlink-testing-framework/tools/citool@83100a879006dde55ace09a5dfd99b37e62f5a3f # v1.34.4
- name: Run Check Tests Command
env:
TEST_PATH: ${{ inputs.test_path }}
CHECK_TEST_PATH: ${{ inputs.check_test_path }}
run: |
if ! citool check-tests ${{ github.workspace }}/${{ inputs.check_test_path }} ${{ github.workspace }}/${{ inputs.test_path }}; then
echo "::error::Some E2E test configurations have to be added to ${{ inputs.test_path }}. This file defines Github CI configuration for each E2E test or set of E2E tests." && exit 1
if ! citool check-tests "${{ github.workspace }}/$CHECK_TEST_PATH" "${{ github.workspace }}/$TEST_PATH"; then
echo "::error::Some E2E test configurations have to be added to $TEST_PATH. This file defines Github CI configuration for each E2E test or set of E2E tests." && exit 1
fi

get_latest_chainlink_release_version:
Expand Down Expand Up @@ -407,6 +422,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/[email protected]
with:
persist-credentials: false
- name: Setup Go
uses: actions/[email protected]
with:
Expand All @@ -422,14 +439,20 @@ jobs:
- name: Generate Docker Tests Matrix
id: set-docker-matrix
shell: bash
env:
CUSTOM_TEST_LIST_JSON: ${{ inputs.custom_test_list_json }}
TEST_PATH: ${{ inputs.test_path }}
TEST_LIST: ${{ inputs.test_list }}
TEST_IDS: ${{ inputs.test_ids }}
TEST_TRIGGER: ${{ inputs.test_trigger }}
run: |
# Check if custom_test_list_json is provided and non-empty
if [[ -n '${{ inputs.custom_test_list_json }}' ]]; then
if [[ -n "$CUSTOM_TEST_LIST_JSON" ]]; then
echo "Using custom test list JSON"
MATRIX_JSON=$(echo '${{ inputs.custom_test_list_json }}' | jq -c '{tests: [.tests[] | select(.test_env_type == "docker")]}')
MATRIX_JSON=$(echo "$CUSTOM_TEST_LIST_JSON" | jq -c '{tests: [.tests[] | select(.test_env_type == "docker")]}')
else
echo "Using default test list"
MATRIX_JSON=$(citool filter --file ${{ github.workspace }}/${{ inputs.test_path }} --test-env-type 'docker' --test-list '${{ inputs.test_list }}' --test-ids '${{ inputs.test_ids }}' --workflow '${{ inputs.test_trigger }}')
MATRIX_JSON=$(citool filter --file "${{ github.workspace }}/$TEST_PATH" --test-env-type 'docker' --test-list "$TEST_LIST" --test-ids "$TEST_IDS" --workflow "$TEST_TRIGGER")
fi

echo "Docker tests:"
Expand All @@ -439,14 +462,20 @@ jobs:
- name: Generate K8s Tests Matrix
id: set-k8s-runner-matrix
shell: bash
env:
CUSTOM_TEST_LIST_JSON: ${{ inputs.custom_test_list_json }}
TEST_PATH: ${{ inputs.test_path }}
TEST_LIST: ${{ inputs.test_list }}
TEST_IDS: ${{ inputs.test_ids }}
TEST_TRIGGER: ${{ inputs.test_trigger }}
run: |
# Check if custom_test_list_json is provided and non-empty
if [[ -n '${{ inputs.custom_test_list_json }}' ]]; then
if [[ -n "$CUSTOM_TEST_LIST_JSON" ]]; then
echo "Using custom test list JSON"
MATRIX_JSON=$(echo '${{ inputs.custom_test_list_json }}' | jq -c '{tests: [.tests[] | select(.test_env_type == "k8s-remote-runner")]}')
MATRIX_JSON=$(echo "$CUSTOM_TEST_LIST_JSON" | jq -c '{tests: [.tests[] | select(.test_env_type == "k8s-remote-runner")]}')
else
echo "Using default test list"
MATRIX_JSON=$(citool filter --file ${{ github.workspace }}/${{ inputs.test_path }} --test-env-type 'k8s-remote-runner' --test-list '${{ inputs.test_list }}' --test-ids '${{ inputs.test_ids }}' --workflow '${{ inputs.test_trigger }}')
MATRIX_JSON=$(citool filter --file "${{ github.workspace }}/$TEST_PATH" --test-env-type 'k8s-remote-runner' --test-list "$TEST_LIST" --test-ids "$TEST_IDS" --workflow "$TEST_TRIGGER")
fi

echo "K8s tests:"
Expand Down Expand Up @@ -486,14 +515,18 @@ jobs:

- name: Check if team is required
if: ${{ steps.check-matrices.outputs.run-k8s-tests == 'true' }}
env:
TEAM: ${{ inputs.team }}
run: |
if [[ -z "${{ inputs.team }}" ]]; then
if [[ -z "$TEAM" ]]; then
echo "Team is required for k8s tests"
exit 1
fi

- name: Check if test secrets are required for any test
shell: bash
env:
TEST_PATH: ${{ inputs.test_path }}
run: |
# Check if the test secret key is provided from GitHub Secrets and skip the checks if it is non-empty
if [[ -n "${{ secrets.TEST_SECRETS_OVERRIDE_BASE64 }}" ]]; then
Expand All @@ -514,7 +547,7 @@ jobs:

# Determine if any tests require secrets
if [ -n "$DOCKER_TESTS_REQUIRING_SECRETS" ] || [ -n "$K8S_TESTS_REQUIRING_SECRETS" ]; then
echo "Tests in ${{ github.workspace }}/${{ inputs.test_path }} requiring custom test secrets:"
echo "Tests in ${{ github.workspace }}/$TEST_PATH requiring custom test secrets:"
if [ -n "$DOCKER_TESTS_REQUIRING_SECRETS" ]; then
echo "$DOCKER_TESTS_REQUIRING_SECRETS"
fi
Expand Down Expand Up @@ -558,6 +591,8 @@ jobs:
steps:
- name: Checkout the repo
uses: actions/[email protected]
with:
persist-credentials: false

- name: Get Chainlink image
uses: ./.github/actions/build-chainlink-image
Expand Down Expand Up @@ -595,6 +630,8 @@ jobs:
steps:
- name: Checkout the repo
uses: actions/[email protected]
with:
persist-credentials: false

- name: Get Chainlink plugins image
uses: ./.github/actions/build-chainlink-image
Expand Down Expand Up @@ -647,6 +684,8 @@ jobs:
uses: catchpoint/workflow-telemetry-action@94c3c3d9567a0205de6da68a76c428ce4e769af1 # v2.0.0
- name: Checkout repository
uses: actions/[email protected]
with:
persist-credentials: false
- name: Install jq
run: sudo apt-get install -y jq

Expand Down Expand Up @@ -916,6 +955,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/[email protected]
with:
persist-credentials: false
- name: Build Test Runner Image
id: build-test-runner-image
uses: smartcontractkit/.github/actions/ctf-build-test-image@main # main branch
Expand All @@ -927,14 +968,17 @@ jobs:
suites: ${{ inputs.test_image_suites }}
- name: Set Remote Runner Version
id: set-remote-runner-version
env:
WITH_EXISTING_REMOTE_RUNNER_VERSION:
${{ inputs.with_existing_remote_runner_version }}
run: |
# shellcheck disable=SC2129
if [[ -z "${{ inputs.with_existing_remote_runner_version }}" ]]; then
if [[ -z "$WITH_EXISTING_REMOTE_RUNNER_VERSION" ]]; then
echo "remote-runner-image=${{ steps.build-test-runner-image.outputs.test_image }}" >> "$GITHUB_OUTPUT"
echo "remote-runner-repository=${{ steps.build-test-runner-image.outputs.test_image_repository }}" >> "$GITHUB_OUTPUT"
echo "remote-runner-version=${{ steps.build-test-runner-image.outputs.test_image_tag }}" >> "$GITHUB_OUTPUT"
else
echo "remote-runner-version=${{ inputs.with_existing_remote_runner_version }}" >> "$GITHUB_OUTPUT"
echo "remote-runner-version=$WITH_EXISTING_REMOTE_RUNNER_VERSION" >> "$GITHUB_OUTPUT"
fi

run-k8s-runner-tests:
Expand Down Expand Up @@ -977,6 +1021,8 @@ jobs:
uses: catchpoint/workflow-telemetry-action@94c3c3d9567a0205de6da68a76c428ce4e769af1 # v2.0.0
- name: Checkout repository
uses: actions/[email protected]
with:
persist-credentials: false
- name: Install jq
run: sudo apt-get install -y jq

Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/run-integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/[email protected]
with:
persist-credentials: false
- name: Setup Go
uses: actions/[email protected]
with:
Expand All @@ -137,9 +139,13 @@ jobs:
- name: Generate Tests Matrix
id: set-matrix
shell: bash
env:
TEST_PATH: ${{ inputs.test_path }}
TEST_IDS: ${{ inputs.test_ids }}
TEST_TRIGGER: ${{ inputs.test_trigger }}
run: |
echo "Using default test list"
MATRIX_JSON=$(citool filter --file ${{ github.workspace }}/${{ inputs.test_path }} --test-env-type 'in-memory' --test-ids '${{ inputs.test_ids }}' --workflow '${{ inputs.test_trigger }}')
MATRIX_JSON=$(citool filter --file "${{ github.workspace }}/$TEST_PATH" --test-env-type 'in-memory' --test-ids "$TEST_IDS" --workflow "$TEST_TRIGGER")
echo "Tests:"
echo "$MATRIX_JSON" | jq
echo "matrix=$MATRIX_JSON" >> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -203,6 +209,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/[email protected]
with:
persist-credentials: false
- name: Wait for postgres to be ready
run: |
until pg_isready -h localhost -p 5432 -U postgres; do
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/schedule-renovate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ jobs:
uses: actions/[email protected]
with:
fetch-depth: 0
persist-credentials: false

- name: setup-renovate
uses: ./actions/setup-renovate
Expand Down
Loading
Loading