Skip to content

Commit

Permalink
Support additional docker build flags
Browse files Browse the repository at this point in the history
This adds support for the following docker flags:

--build-arg: Pass build arguments that can be used in the template
dockerfile
--ssh: Provide a ssh configuration to the container while building
--secret: Provide a secret to the container while building
--no-cache: Build the image and do not use cached layers.

It also fixes how we call the docker buildx plugin so that user
configuration, such as a proxy, is used.

After upgrading to a new version of buildx, I was also able to pick up a
fix for pretty printing the progress to stderr, while capturing the
plaintext output to the logs.

Closes getporter#1769
Closes getporter#1941

Signed-off-by: Carolyn Van Slyck <[email protected]>
  • Loading branch information
carolynvs committed Apr 1, 2022
1 parent b5b2e43 commit bf8c94a
Show file tree
Hide file tree
Showing 27 changed files with 434 additions and 285 deletions.
10 changes: 9 additions & 1 deletion cmd/porter/bundle.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,15 @@ Porter uses the docker driver as the default build driver, an alternate driver m
"Path to the build context directory where all bundle assets are located.")
f.StringVar(&opts.Driver, "driver", porter.BuildDriverDefault,
fmt.Sprintf("Experimental. Driver for building the invocation image. Allowed values are: %s", strings.Join(porter.BuildDriverAllowedValues, ", ")))

f.StringSliceVar(&opts.SSH, "ssh", nil,
f.StringArrayVar(&opts.BuildArgs, "build-arg", nil,
"Set build arguments in the template Dockerfile (format: NAME=VALUE). May be specified multiple times.")
f.StringArrayVar(&opts.SSH, "ssh", nil,
"SSH agent socket or keys to expose to the build (format: default|<id>[=<socket>|<key>[,<key>]]). May be specified multiple times.")
f.StringArrayVar(&opts.Secrets, "secret", nil,
"Secret file to expose to the build (format: id=mysecret,src=/local/secret). May be specified multiple times.")
f.BoolVar(&opts.NoCache, "no-cache", false,
"Do not use cache when building the image.")
// Allow configuring the --driver flag with build-driver, to avoid conflicts with other commands
cmd.Flag("driver").Annotations = map[string][]string{
"viper-key": {"build-driver"},
Expand Down
20 changes: 12 additions & 8 deletions docs/content/cli/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,18 @@ porter build [flags]
### Options

```
-d, --dir string Path to the build context directory where all bundle assets are located.
--driver string Experimental. Driver for building the invocation image. Allowed values are: docker, buildkit (default "docker")
-f, --file porter.yaml Path to the Porter manifest. Defaults to porter.yaml in the current directory.
-h, --help help for build
--name string Override the bundle name
--no-lint Do not run the linter
-v, --verbose Enable verbose logging
--version string Override the bundle version
--driver string Experimental. Driver for building the invocation image. Allowed values are: buildkit (default "buildkit")
--build-arg stringArray Set build arguments in the template Dockerfile (format: NAME=VALUE). May be specified multiple times.
-d, --dir string Path to the build context directory where all bundle assets are located.
-f, --file porter.yaml Path to the Porter manifest. Defaults to porter.yaml in the current directory.
-h, --help help for build
--name string Override the bundle name
--no-cache Do not use cache when building the image.
--no-lint Do not run the linter
--secret stringArray Secret file to expose to the build (format: id=mysecret,src=/local/secret). May be specified multiple times.
--ssh stringArray SSH agent socket or keys to expose to the build (format: default|<id>[=<socket>|<key>[,<key>]]). May be specified multiple times.
-v, --verbose Enable verbose logging
--version string Override the bundle version
```

### Options inherited from parent commands
Expand Down
20 changes: 12 additions & 8 deletions docs/content/cli/bundles_build.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,18 @@ porter bundles build [flags]
### Options

```
-d, --dir string Path to the build context directory where all bundle assets are located.
--driver string Experimental. Driver for building the invocation image. Allowed values are: docker, buildkit (default "docker")
-f, --file porter.yaml Path to the Porter manifest. Defaults to porter.yaml in the current directory.
-h, --help help for build
--name string Override the bundle name
--no-lint Do not run the linter
-v, --verbose Enable verbose logging
--version string Override the bundle version
--driver string Experimental. Driver for building the invocation image. Allowed values are: buildkit (default "buildkit")
--build-arg stringArray Set build arguments in the template Dockerfile (format: NAME=VALUE). May be specified multiple times.
-d, --dir string Path to the build context directory where all bundle assets are located.
-f, --file porter.yaml Path to the Porter manifest. Defaults to porter.yaml in the current directory.
-h, --help help for build
--name string Override the bundle name
--no-cache Do not use cache when building the image.
--no-lint Do not run the linter
--secret stringArray Secret file to expose to the build (format: id=mysecret,src=/local/secret). May be specified multiple times.
--ssh stringArray SSH agent socket or keys to expose to the build (format: default|<id>[=<socket>|<key>[,<key>]]). May be specified multiple times.
-v, --verbose Enable verbose logging
--version string Override the bundle version
```

### Options inherited from parent commands
Expand Down
5 changes: 5 additions & 0 deletions examples/private-assets/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# See https://docs.docker.com/engine/reference/builder/#dockerignore-file
# Put files here that you don't want copied into your bundle's invocation image
.gitignore
Dockerfile.tmpl
secrets/
1 change: 1 addition & 0 deletions examples/private-assets/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.cnab/
33 changes: 33 additions & 0 deletions examples/private-assets/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Bundle with Private Assets

Sometimes you need to include assets from secured locations, such as a private repository in your bundle.
You can use the \--secret flag to pass secrets into the bundle when it is built.

## Try it out
1. Edit secrets/token and replace the contents with a [GitHub Personal Access Token](https://github.com/settings/tokens).
The permissions do not matter for this sample bundle.
There should not be a newline at the end of the file.

1. Build the bundle and pass the secret into the bundle with \--secret
```
porter build --secret id=token,src=secrets/token
```
1. Install the bundle to see the private assets embedded in the bundle
```
$ porter install example-private-assets --reference ghcr.io/getporter/examples/private-assets:v0.1.0
__________________________
< yarr, I'm a secret whale >
--------------------------
\
\
\
## .
## ## ## ==
## ## ## ## ===
/""""""""""""""""___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\______/
```
7 changes: 7 additions & 0 deletions examples/private-assets/check-secrets.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail

if [[ ! -f "/run/secrets/token" ]]; then
echo "You forgot to use --secret id=token,src=secrets/token"
exit 1
fi
17 changes: 17 additions & 0 deletions examples/private-assets/helpers.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail

install() {
echo Hello World
}

upgrade() {
echo World 2.0
}

uninstall() {
echo Goodbye World
}

# Call the requested function and pass the arguments as-is
"$@"
28 changes: 28 additions & 0 deletions examples/private-assets/porter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
schemaVersion: 1.0.0-alpha.1

name: private-assets
version: 0.1.0
description: "Example bundle that contains private assets and prints it when run"
registry: ghcr.io/getporter/examples/
dockerfile: template.Dockerfile

mixins:
- exec

install:
- exec:
command: cat
arguments:
- /secret

upgrade:
- exec:
command: cat
arguments:
- /secret

uninstall:
- exec:
command: cat
arguments:
- /secret
1 change: 1 addition & 0 deletions examples/private-assets/secrets/token
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
REPLACE_WITH_YOUR_GITHUB_TOKEN
19 changes: 19 additions & 0 deletions examples/private-assets/template.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# syntax=docker/dockerfile-upstream:1.4.0-rc2
FROM debian:stretch-slim

# PORTER_INIT

RUN rm -f /etc/apt/apt.conf.d/docker-clean; echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache
RUN --mount=type=cache,target=/var/cache/apt --mount=type=cache,target=/var/lib/apt \
apt-get update && apt-get install -y ca-certificates curl

# PORTER_MIXINS

# Use the BUNDLE_DIR build argument to copy files into the bundle's working directory
COPY --link . ${BUNDLE_DIR}

# Check the secret was passed to the build command
RUN --mount=type=secret,id=token /cnab/app/check-secrets.sh

# Use the injected secrets to build private assets into the bundle
RUN --mount=type=secret,id=token curl -O https://$(cat /run/secrets/token)@gist.githubusercontent.com/carolynvs/860a0d26de3af1468d290a075a91aac9/raw/c53223acd284830e8f541cf35eba94dde0ddf75d/secret
59 changes: 32 additions & 27 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ module get.porter.sh/porter
go 1.18

replace (
github.com/docker/cli => github.com/docker/cli v20.10.3-0.20220226190722-8667ccd1124c+incompatible
github.com/docker/docker => github.com/docker/docker v20.10.3-0.20220121014307-40bb9831756f+incompatible
// See https://github.com/hashicorp/go-plugin/pull/127 and
// https://github.com/hashicorp/go-plugin/pull/163
// Also includes a branch we haven't PR'd yet: capture-yamux-logs
Expand All @@ -25,13 +27,13 @@ require (
github.com/cnabio/cnab-to-oci v0.3.3
github.com/containerd/console v1.0.3
github.com/containerd/containerd v1.6.1
github.com/docker/buildx v0.5.1
github.com/docker/buildx v0.8.1
github.com/docker/cli v20.10.13+incompatible
github.com/docker/distribution v2.8.1+incompatible
github.com/docker/docker v20.10.13+incompatible
github.com/dustin/go-humanize v1.0.0
github.com/ghodss/yaml v1.0.0
github.com/google/go-cmp v0.5.6
github.com/google/go-cmp v0.5.7
github.com/google/go-containerregistry v0.5.1
github.com/hashicorp/go-hclog v0.14.1
github.com/hashicorp/go-multierror v1.1.1
Expand All @@ -40,14 +42,13 @@ require (
github.com/mattn/go-colorable v0.1.7
github.com/mattn/go-isatty v0.0.12
github.com/mikefarah/yq/v3 v3.0.0-20201020025845-ccb718cd0f59
github.com/mitchellh/mapstructure v1.3.3
github.com/mitchellh/mapstructure v1.4.2
github.com/mmcdole/gofeed v1.0.0-beta2
github.com/moby/buildkit v0.9.0
github.com/moby/term v0.0.0-20210610120745-9d4ed1856297
github.com/moby/buildkit v0.10.0
github.com/olekukonko/tablewriter v0.0.4
github.com/opencontainers/go-digest v1.0.0
github.com/osteele/liquid v1.3.0
github.com/pelletier/go-toml v1.9.3
github.com/pelletier/go-toml v1.9.4
github.com/pivotal/image-relocation v0.0.0-20191111101224-e94aff6df06c
github.com/pkg/errors v0.9.1
github.com/spf13/afero v1.5.1
Expand All @@ -57,12 +58,12 @@ require (
github.com/stretchr/testify v1.7.0
github.com/xeipuuv/gojsonschema v1.2.0
go.mongodb.org/mongo-driver v1.7.1
go.opentelemetry.io/otel v1.3.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.3.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.3.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.3.0
go.opentelemetry.io/otel/sdk v1.3.0
go.opentelemetry.io/otel/trace v1.3.0
go.opentelemetry.io/otel v1.4.1
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.4.1
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.4.1
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.4.1
go.opentelemetry.io/otel/sdk v1.4.1
go.opentelemetry.io/otel/trace v1.4.1
go.uber.org/zap v1.17.0
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
google.golang.org/grpc v1.45.0
Expand All @@ -82,7 +83,6 @@ require (
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/Masterminds/semver v1.5.0 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/Microsoft/hcsshim v0.9.2 // indirect
github.com/PaesslerAG/gval v1.0.0 // indirect
github.com/PuerkitoBio/goquery v1.5.0 // indirect
github.com/agl/ed25519 v0.0.0-20170116200512-5312a6153412 // indirect
Expand All @@ -91,20 +91,21 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.1.2 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/containerd/cgroups v1.0.3 // indirect
github.com/containerd/continuity v0.2.2 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.6.4 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.11.2 // indirect
github.com/containerd/ttrpc v1.1.0 // indirect
github.com/containerd/typeurl v1.0.2 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20210303052042-6bc126869bf4 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/docker-credential-helpers v0.6.3 // indirect
github.com/docker/docker-credential-helpers v0.6.4 // indirect
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-metrics v0.0.1 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/dsnet/compress v0.0.1 // indirect
github.com/fatih/color v1.9.0 // indirect
github.com/felixge/httpsnoop v1.0.2 // indirect
github.com/form3tech-oss/jwt-go v3.2.3+incompatible // indirect
github.com/fsnotify/fsnotify v1.4.9 // indirect
github.com/fvbommel/sortorder v1.0.2 // indirect
Expand All @@ -115,7 +116,6 @@ require (
github.com/gofrs/flock v0.7.3 // indirect
github.com/gogo/googleapis v1.4.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.4-0.20210608040537-544b4180ac70 // indirect
github.com/google/gofuzz v1.2.0 // indirect
Expand Down Expand Up @@ -146,9 +146,10 @@ require (
github.com/mitchellh/reflectwalk v1.0.0 // indirect
github.com/mmcdole/goxpp v0.0.0-20181012175147-0068e33feabf // indirect
github.com/moby/locker v1.0.1 // indirect
github.com/moby/sys/mount v0.2.0 // indirect
github.com/moby/sys/mount v0.3.0 // indirect
github.com/moby/sys/mountinfo v0.6.0 // indirect
github.com/moby/sys/symlink v0.2.0 // indirect
github.com/moby/sys/signal v0.6.0 // indirect
github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
Expand All @@ -160,9 +161,9 @@ require (
github.com/osteele/tuesday v1.0.3 // indirect
github.com/pierrec/lz4/v4 v4.0.3 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.11.0 // indirect
github.com/prometheus/client_golang v1.12.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.30.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.7.3 // indirect
github.com/qri-io/jsonpointer v0.1.1 // indirect
github.com/qri-io/jsonschema v0.2.2-0.20210831022256-780655b2ba0e // indirect
Expand All @@ -173,10 +174,11 @@ require (
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/subosito/gotenv v1.2.0 // indirect
github.com/theupdateframework/notary v0.6.1 // indirect
github.com/tonistiigi/fsutil v0.0.0-20210609172227-d72af97c0eaf // indirect
github.com/tonistiigi/fsutil v0.0.0-20220315205639-9ed612626da3 // indirect
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea // indirect
github.com/tonistiigi/vt100 v0.0.0-20210615222946-8066bb97264f // indirect
github.com/ulikunitz/xz v0.5.7 // indirect
github.com/vbatts/tar-split v0.11.2 // indirect
github.com/weppos/publicsuffix-go v0.5.0 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.0.2 // indirect
Expand All @@ -187,13 +189,16 @@ require (
github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d // indirect
github.com/zmap/zcrypto v0.0.0-20190729165852-9051775e6a2e // indirect
github.com/zmap/zlint v0.0.0-20190806154020-fd021b4cfbeb // indirect
go.opencensus.io v0.23.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.28.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.3.0 // indirect
go.opentelemetry.io/proto/otlp v0.11.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.29.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.29.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.29.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.4.1 // indirect
go.opentelemetry.io/otel/internal/metric v0.27.0 // indirect
go.opentelemetry.io/otel/metric v0.27.0 // indirect
go.opentelemetry.io/proto/otlp v0.12.0 // indirect
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.6.0 // indirect
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
golang.org/x/crypto v0.0.0-20211202192323-5770296d904e // indirect
golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect
golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5 // indirect
Expand Down
Loading

0 comments on commit bf8c94a

Please sign in to comment.