Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Java artifacts scanning from vulnerability scans [DI-50] #778

Merged
Merged
Changes from 34 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
5fe7f98
Create empty ZIP
JackPGreen Jul 4, 2024
9ad4e96
WIP
JackPGreen Jul 4, 2024
9652ba3
WIP
JackPGreen Jul 4, 2024
530b05a
WIP
JackPGreen Jul 4, 2024
9a7a94f
WIP
JackPGreen Jul 4, 2024
28582f6
WIP
JackPGreen Jul 4, 2024
c7cfb7d
WIP
JackPGreen Jul 4, 2024
20e0900
WIP
JackPGreen Jul 4, 2024
90498d6
WIP
JackPGreen Jul 4, 2024
1d4dd1d
WIP
JackPGreen Jul 4, 2024
ad2aaca
WIP
JackPGreen Jul 4, 2024
4277702
WIP
JackPGreen Jul 4, 2024
ec7584c
WIP
JackPGreen Jul 4, 2024
9beec7b
Refactor
JackPGreen Jul 4, 2024
22fac1e
make temp directory
JackPGreen Jul 4, 2024
e58c0cb
Revert
JackPGreen Jul 4, 2024
66bbc4e
WIP
JackPGreen Jul 4, 2024
a67446b
WIP
JackPGreen Jul 4, 2024
82eda87
WIP
JackPGreen Jul 4, 2024
fcc20af
WIP
JackPGreen Jul 4, 2024
61b67fe
WIP
JackPGreen Jul 4, 2024
2d7050c
WIP
JackPGreen Jul 4, 2024
3948e73
REVERT
JackPGreen Jul 4, 2024
111353d
WIP
JackPGreen Jul 4, 2024
ca3260f
WIP
JackPGreen Jul 4, 2024
b71b62c
WIP
JackPGreen Jul 4, 2024
bd10914
WIP
JackPGreen Jul 4, 2024
38d8bb1
WIP
JackPGreen Jul 4, 2024
49a80bd
REVERT
JackPGreen Jul 4, 2024
bd3d829
REVERT
JackPGreen Jul 4, 2024
a1f46a9
Merge branch 'master' into DI-50---Remove-java-artifacts-scanning-fro…
JackPGreen Jul 4, 2024
d67ad1a
Update vulnerability_scan_subworkflow.yml
JackPGreen Jul 4, 2024
b8ae9ab
Update vulnerability_scan_subworkflow.yml
JackPGreen Jul 4, 2024
5978b34
Use HZ fork
JackPGreen Jul 8, 2024
12a1d84
Merge branch 'master' into DI-50---Remove-java-artifacts-scanning-fro…
JackPGreen Jul 9, 2024
e3907bf
Merge branch 'master' into DI-50---Remove-java-artifacts-scanning-fro…
JackPGreen Jul 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 47 additions & 83 deletions .github/workflows/vulnerability_scan_subworkflow.yml
Original file line number Diff line number Diff line change
@@ -1,78 +1,33 @@
name: Vulnerability Scan

on:
workflow_call:
inputs:
ref:
required: true
type: string
secrets:
SNYK_TOKEN:
required: true
workflow_dispatch:
inputs:
ref:
required: true
type: string
workflow_call:
inputs:
ref:
required: true
type: string
secrets:
SNYK_TOKEN:
required: true
workflow_dispatch:
inputs:
ref:
required: true
type: string

jobs:
scan-oss:
env:
DOCKLE_HOST: "unix:///var/run/docker.sock"
scan:
runs-on: ubuntu-latest
steps:
- name: Checkout Code at ${{ inputs.ref }} branch
uses: actions/checkout@v4
with:
ref: ${{ inputs.ref }}

- name: Install xmllint
uses: ./.github/actions/install-xmllint

- name: Get OSS dist ZIP
run: |
. .github/scripts/oss-build.functions.sh
HZ_VERSION=$(awk -F '=' '/^ARG HZ_VERSION=/ {print $2}' hazelcast-oss/Dockerfile)
export HZ_SNAPSHOT_INTERNAL_PASSWORD=${{ secrets.HZ_SNAPSHOT_INTERNAL_PASSWORD }}
export HZ_SNAPSHOT_INTERNAL_USERNAME=${{ secrets.HZ_SNAPSHOT_INTERNAL_USERNAME }}
HAZELCAST_OSS_ZIP_URL=$(get_hz_dist_zip "" "${HZ_VERSION}")
curl --fail --silent --show-error --location "$HAZELCAST_OSS_ZIP_URL" --output hazelcast-oss/hazelcast-distribution.zip;

- name: Build OSS image
run: |
docker build -t hazelcast/oss:${{ github.sha }} hazelcast-oss

- name: Scan OSS image by Trivy
if: always()
uses: aquasecurity/[email protected]
with:
image-ref: hazelcast/oss:${{ github.sha }}
trivy-config: .github/containerscan/trivy.yaml

- name: Scan OSS image by Dockle
if: always()
uses: goodwithtech/dockle-action@main
with:
image: hazelcast/oss:${{ github.sha }}
format: 'list'
exit-code: '1'
exit-level: 'warn'
# too many false positives, we don't use credentials in Dockerfile
ignore: 'CIS-DI-0010'

- name: Scan OSS image by Snyk
if: always()
uses: snyk/actions/docker@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
image: hazelcast/oss:${{ github.sha }}
args: --file=hazelcast-oss/Dockerfile --policy-path=.github/containerscan --severity-threshold=high --exclude-base-image-vulns

scan-ee:
strategy:
matrix:
image:
- label: oss
distribution_zip_name: hazelcast-distribution.zip
- label: enterprise
nishaatr marked this conversation as resolved.
Show resolved Hide resolved
distribution_zip_name: hazelcast-enterprise-distribution.zip
env:
DOCKLE_HOST: "unix:///var/run/docker.sock"
ldziedziul marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-latest
DIRECTORY: hazelcast-${{ matrix.image.label }}
IMAGE_TAG: hazelcast/${{ matrix.image.label }}:${{ github.sha }}
steps:
- name: Checkout Code at ${{ inputs.ref }} branch
uses: actions/checkout@v4
Expand All @@ -82,40 +37,49 @@ jobs:
- name: Install xmllint
uses: ./.github/actions/install-xmllint

- name: Get EE dist ZIP
- name: Generate ${{ matrix.image.label }} dist ZIP
run: |
. .github/scripts/ee-build.functions.sh
HZ_VERSION=$(awk -F '=' '/^ARG HZ_VERSION=/ {print $2}' hazelcast-enterprise/Dockerfile)
HAZELCAST_EE_ZIP_URL=$(get_hz_dist_zip "" "${HZ_VERSION}")
curl --fail --silent --show-error --location "$HAZELCAST_EE_ZIP_URL" --output hazelcast-enterprise/hazelcast-enterprise-distribution.zip;
# Make a dummy empty ZIP file to avoid scanning Java dependencies, as managed downstream
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use the real zip but remove all *.jar files from it? This way we would be a bit closer to the docker image we actually push.

I have concerns about executable scripts from the distribution zips, if they're used in a safe manner, although I'm not sure if they're scanned.

WDYT @kwart ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use the real zip but remove all *.jar files from it? This way we would be a bit closer to the docker image we actually push.

I have concerns about executable scripts from the distribution zips, if they're used in a safe manner, although I'm not sure if they're scanned.

WDYT @kwart ?

I'm assuming that the distribution ZIPs are already scanned and considered safe, so further tests are redundant/duplicated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jars are scanned for sure, not sure about shell scripts. I don't have strong opinion here.
Don't know if it makes any difference, if you compare the scan results it might be an indicator it it makes sense

# DI-50 - Remove java artifacts scanning from hazelcast-docker
working_directory=hazelcast-distribution
mkdir -p ${working_directory}/lib
nishaatr marked this conversation as resolved.
Show resolved Hide resolved
mkdir -p ${working_directory}/bin
touch ${working_directory}/bin/empty
nishaatr marked this conversation as resolved.
Show resolved Hide resolved

- name: Build EE image
zip -r ${{ env.DIRECTORY }}/${{ matrix.image.distribution_zip_name }} ${working_directory}

- name: Build ${{ matrix.image.label }} image
run: |
docker build -t hazelcast/ee:${{ github.sha }} hazelcast-enterprise
docker build -t ${{ env.IMAGE_TAG }} ${{ env.DIRECTORY }}

- name: Scan EE image by Trivy
- name: Scan ${{ matrix.image.label }} image by Trivy
if: always()
uses: aquasecurity/[email protected]
with:
image-ref: hazelcast/ee:${{ github.sha }}
image-ref: ${{ env.IMAGE_TAG }}
trivy-config: .github/containerscan/trivy.yaml
env:
# https://github.com/aquasecurity/trivy/issues/2432
DOCKLE_HOST: "unix:///var/run/docker.sock"

- name: Scan EE image by Dockle
- name: Scan ${{ matrix.image.label }} image by Dockle
if: always()
uses: goodwithtech/dockle-action@main
# Use our fork until https://github.com/goodwithtech/dockle-action/issues/7 is fixed
# uses: goodwithtech/dockle-action@main
uses: hazelcast/dockle-action/@Upgrade-Dockle-to-`0.4.14`
with:
image: hazelcast/ee:${{ github.sha }}
image: ${{ env.IMAGE_TAG }}
format: 'list'
exit-code: '1'
exit-level: 'warn'
# too many false positives, we don't use credentials in Dockerfile
ignore: 'CIS-DI-0010'

- name: Scan EE image by Snyk
- name: Scan ${{ matrix.image.label }} image by Snyk
if: always()
uses: snyk/actions/docker@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
image: hazelcast/ee:${{ github.sha }}
args: --file=hazelcast-enterprise/Dockerfile --policy-path=.github/containerscan --severity-threshold=high --exclude-base-image-vulns
image: ${{ env.IMAGE_TAG }}
args: --file=${{ env.DIRECTORY }}/Dockerfile --policy-path=.github/containerscan --severity-threshold=high --exclude-base-image-vulns