[WIP] images/releng/ko-builder: Initial commit

[justaugustus]

What type of PR is this?

/kind feature

What this PR does / why we need it:

This is inspired by the Tekton ko build image (h/t @priyawadhwa): https://github.com/tektoncd/plumbing/blob/fe57aef52b996fa58ca2ff5529819c8706f2ac4b/tekton/images/ko-gcloud/Dockerfile

Here we attempt to provide a simpler means of building container images for the Golang applications we maintain via ko.

Signed-off-by: Stephen Augustus [email protected]

Which issue(s) this PR fixes:

Special notes for your reviewer:

This is likely missing a few things, but just getting it off of my local and in public early.

Does this PR introduce a user-facing change?

- Initial build of ko-builder image

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justaugustus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• images/OWNERS [justaugustus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[justaugustus]

Slack thread: https://kubernetes.slack.com/archives/CJH2GBF7Y/p1622038124127700
cc: @kubernetes/release-engineering

Here's the current state:

❯ time docker run -it -v $(pwd):/workspace gcr.io/k8s-staging-releng/ko-builder publish ./cmd/cip --local
2021/05/26 13:42:09 Using base gcr.io/distroless/static:nonroot for sigs.k8s.io/k8s-container-image-promoter/cmd/cip
2021/05/26 13:42:10 Building sigs.k8s.io/k8s-container-image-promoter/cmd/cip for linux/amd64
2021/05/26 13:42:43 Loading ko.local/cip-d9236763b0fdb29506a7a9abe4767208:b5a2ad2ce67333203fa09c3d4f35bf20ac1cc958d0b2b0935555a1c4965e136e
Error: failed to publish images: error publishing ko://sigs.k8s.io/k8s-container-image-promoter/cmd/cip: error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Usage:
  ko publish IMPORTPATH... [flags]

Examples:

  # Build and publish import path references to a Docker
  # Registry as:
  #   ${KO_DOCKER_REPO}/<package name>-<hash of import path>
  # When KO_DOCKER_REPO is ko.local, it is the same as if
  # --local and --preserve-import-paths were passed.
  ko publish github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah

  # Build and publish a relative import path as:
  #   ${KO_DOCKER_REPO}/<package name>-<hash of import path>
  # When KO_DOCKER_REPO is ko.local, it is the same as if
  # --local and --preserve-import-paths were passed.
  ko publish ./cmd/blah

  # Build and publish a relative import path as:
  #   ${KO_DOCKER_REPO}/<import path>
  # When KO_DOCKER_REPO is ko.local, it is the same as if
  # --local was passed.
  ko publish --preserve-import-paths ./cmd/blah

  # Build and publish import path references to a Docker
  # daemon as:
  #   ko.local/<import path>
  # This always preserves import paths.
  ko publish --local github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah

Flags:
      --bare                     Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
  -B, --base-import-paths        Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
      --disable-optimizations    Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
  -h, --help                     help for publish
      --image-label strings      Which labels (key=value) to add to the image.
      --insecure-registry        Whether to skip TLS verification on the registry
  -j, --jobs int                 The maximum number of concurrent builds (default 6)
  -L, --local                    Load into images to local docker daemon.
      --oci-layout-path string   Path to save the OCI image layout of the built images
      --platform string          Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
  -P, --preserve-import-paths    Whether to preserve the full import path after KO_DOCKER_REPO.
      --push                     Push images to KO_DOCKER_REPO (default true)
  -t, --tags strings             Which tags to use for the produced image instead of the default 'latest' tag (may not work properly with --base-import-paths or --bare). (default [latest])
      --tarball string           File to save images tarballs

2021/05/26 13:42:44 error during command execution:failed to publish images: error publishing ko://sigs.k8s.io/k8s-container-image-promoter/cmd/cip: error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
docker run -it -v $(pwd):/workspace gcr.io/k8s-staging-releng/ko-builder     0.12s user 0.11s system 0% cpu 58.148 total

[puerco]

/cc @wilsonehusin to get his insight on recent attempts.

[justaugustus]

Some things to note from above:

• this was run locally
• it's a incredibly rough attempt against the container image promoter repo (in the hopes of eventually removing bazel: [Umbrella] Repo improvements kubernetes-sigs/promo-tools#266)

Known issues:

• needs to be able to auth to GCR within GCB

[imjasonh]

Error: failed to publish images: error publishing ko://sigs.k8s.io/k8s-container-image-promoter/cmd/cip: error loading image: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

This indicates that ko can't reach the Docker socket, which makes sense because you're not mounting the Docker socket in on your docker run.

Do you expect to need to build and publish to a Docker daemon before (presumably) pushing the image to a registry? If not, you can remove the --local and it'll just push directly. If you do need --local, add -v /var/run/docker.sock:/var/run/docker.sock (in general this should be avoided) and the publish should work, I think.

[justaugustus]

@imjasonh -- Thanks for the tip on --local!

Looking better now (outside of the expected auth failure):

❯ time docker run -it -v $(pwd):/workspace -e KO_DOCKER_REPO=gcr.io/k8s-staging-releng-test gcr.io/k8s-staging-releng/ko-builder publish ./cmd/cip
2021/05/26 14:55:55 No matching credentials were found, falling back on anonymous
2021/05/26 14:56:13 Using base gcr.io/distroless/static:nonroot for sigs.k8s.io/k8s-container-image-promoter/cmd/cip
2021/05/26 14:56:14 Building sigs.k8s.io/k8s-container-image-promoter/cmd/cip for linux/amd64
2021/05/26 14:57:13 Publishing gcr.io/k8s-staging-releng-test/cip-d9236763b0fdb29506a7a9abe4767208:latest
2021/05/26 14:57:13 No matching credentials were found for "gcr.io"
Error: failed to publish images: error publishing ko://sigs.k8s.io/k8s-container-image-promoter/cmd/cip: GET https://gcr.io/v2/token?scope=repository%3Ak8s-staging-releng-test%2Fcip-d9236763b0fdb29506a7a9abe4767208%3Apush%2Cpull&scope=repository%3Adistroless%2Fstatic%3Apull&service=gcr.io: UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
Usage:
  ko publish IMPORTPATH... [flags]

Examples:

  # Build and publish import path references to a Docker
  # Registry as:
  #   ${KO_DOCKER_REPO}/<package name>-<hash of import path>
  # When KO_DOCKER_REPO is ko.local, it is the same as if
  # --local and --preserve-import-paths were passed.
  ko publish github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah

  # Build and publish a relative import path as:
  #   ${KO_DOCKER_REPO}/<package name>-<hash of import path>
  # When KO_DOCKER_REPO is ko.local, it is the same as if
  # --local and --preserve-import-paths were passed.
  ko publish ./cmd/blah

  # Build and publish a relative import path as:
  #   ${KO_DOCKER_REPO}/<import path>
  # When KO_DOCKER_REPO is ko.local, it is the same as if
  # --local was passed.
  ko publish --preserve-import-paths ./cmd/blah

  # Build and publish import path references to a Docker
  # daemon as:
  #   ko.local/<import path>
  # This always preserves import paths.
  ko publish --local github.com/foo/bar/cmd/baz github.com/foo/bar/cmd/blah

Flags:
      --bare                     Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
  -B, --base-import-paths        Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
      --disable-optimizations    Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
  -h, --help                     help for publish
      --image-label strings      Which labels (key=value) to add to the image.
      --insecure-registry        Whether to skip TLS verification on the registry
  -j, --jobs int                 The maximum number of concurrent builds (default 6)
  -L, --local                    Load into images to local docker daemon.
      --oci-layout-path string   Path to save the OCI image layout of the built images
      --platform string          Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
  -P, --preserve-import-paths    Whether to preserve the full import path after KO_DOCKER_REPO.
      --push                     Push images to KO_DOCKER_REPO (default true)
  -t, --tags strings             Which tags to use for the produced image instead of the default 'latest' tag (may not work properly with --base-import-paths or --bare). (default [latest])
      --tarball string           File to save images tarballs

2021/05/26 14:57:14 error during command execution:failed to publish images: error publishing ko://sigs.k8s.io/k8s-container-image-promoter/cmd/cip: GET https://gcr.io/v2/token?scope=repository%3Ak8s-staging-releng-test%2Fcip-d9236763b0fdb29506a7a9abe4767208%3Apush%2Cpull&scope=repository%3Adistroless%2Fstatic%3Apull&service=gcr.io: UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
docker run -it -v $(pwd):/workspace -e  gcr.io/k8s-staging-releng/ko-builder   0.15s user 0.20s system 0% cpu 1:37.35 total

[justaugustus]

(No changes yet; just a rebase.)

[k8s-ci-robot]

@justaugustus: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Rerun command
pull-release-image-kpromo	e3a1323	link	/test pull-release-image-kpromo

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[justaugustus]

Closing this out due to bandwidth.
@kubernetes/release-engineering -- I'll leave the branch around in case someone wants to pick this up.
/close

[k8s-ci-robot]

@justaugustus: Closed this PR.

In response to this:

Closing this out due to bandwidth.
@kubernetes/release-engineering -- I'll leave the branch around in case someone wants to pick this up.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[imjasonh]

FYI over in ko-land we're working on providing an official container image at ghcr.io/google/ko, to address the long-standing ko-build/ko#3.

It's not ready for prime-time yet -- it's built from HEAD, and it doesn't include any support for cred helpers -- but if you're curious you can try it out:

docker run --rm \
  --volume $(pwd):/src \
  --workdir /src \
  --volume ~/.docker:/root/.docker \
  -e KO_DOCKER_REPO=registry.example \
  ghcr.io/google/ko \
  build ./

Since this doesn't support cred helpers yet, that ☝️ assumes you have basic username/password auth in your ~/.docker/config.json, which is in general not a great idea.

Or, in a cloudbuild.yaml:

steps:
- name: ghcr.io/google/ko
  args: ['build', './']

(In GCB, a short-lived access token is automatically made available and kept refreshed in ~/.docker/config.json, so this should work even without cred helpers.)