Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/master'
Browse files Browse the repository at this point in the history
  • Loading branch information
wripley committed Oct 29, 2019
2 parents f38c47d + d4badf3 commit 08d0948
Show file tree
Hide file tree
Showing 110 changed files with 19,319 additions and 362 deletions.
1 change: 1 addition & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ go_import_path: github.com/GoogleContainerTools/kaniko

script:
- make test
- make images
32 changes: 30 additions & 2 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Gopkg.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,7 @@ required = [
[[constraint]]
name = "github.com/minio/HighwayHash"
version = "1.0.0"

[[constraint]]
name = "github.com/Azure/azure-storage-blob-go"
version = "0.8.0"
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ GO_LDFLAGS += '
EXECUTOR_PACKAGE = $(REPOPATH)/cmd/executor
WARMER_PACKAGE = $(REPOPATH)/cmd/warmer
KANIKO_PROJECT = $(REPOPATH)/kaniko
BUILD_ARG ?= ''
BUILD_ARG ?=

out/executor: $(GO_FILES)
GOARCH=$(GOARCH) GOOS=linux CGO_ENABLED=0 go build -ldflags $(GO_LDFLAGS) -o $@ $(EXECUTOR_PACKAGE)
Expand Down
113 changes: 67 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,48 +18,53 @@ _If you are interested in contributing to kaniko, see [DEVELOPMENT.md](DEVELOPME
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [How does kaniko work?](#how-does-kaniko-work)
- [Known Issues](#known-issues)
- [Demo](#demo)
- [Tutorial](#tutorial)
- [Using kaniko](#using-kaniko)
- [kaniko Build Contexts](#kaniko-build-contexts)
- [Running kaniko](#running-kaniko)
- [Running kaniko in a Kubernetes cluster](#running-kaniko-in-a-kubernetes-cluster)
- [Kubernetes secret](#kubernetes-secret)
- [Running kaniko in gVisor](#running-kaniko-in-gvisor)
- [Running kaniko in Google Cloud Build](#running-kaniko-in-google-cloud-build)
- [Running kaniko in Docker](#running-kaniko-in-docker)
- [Caching](#caching)
- [Caching Layers](#caching-layers)
- [Caching Base Images](#caching-base-images)
- [Pushing to Different Registries](#pushing-to-different-registries)
- [Pushing to Amazon ECR](#pushing-to-amazon-ecr)
- [Additional Flags](#additional-flags)
- [--build-arg](#--build-arg)
- [--cache](#--cache)
- [--cache-dir](#--cache-dir)
- [--cache-repo](#--cache-repo)
- [--cleanup](#--cleanup)
- [--digest-file](#--digest-file)
- [--insecure](#--insecure)
- [--insecure-pull](#--insecure-pull)
- [--no-push](#--no-push)
- [--oci-layout-path](#--oci-layout-path)
- [--reproducible](#--reproducible)
- [--single-snapshot](#--single-snapshot)
- [--snapshotMode](#--snapshotmode)
- [--skip-tls-verify](#--skip-tls-verify)
- [--skip-tls-verify-pull](#--skip-tls-verify-pull)
- [--target](#--target)
- [--tarPath](#--tarpath)
- [--verbosity](#--verbosity)
- [Debug Image](#debug-image)
- [Security](#security)
- [Comparison with Other Tools](#comparison-with-other-tools)
- [Community](#community)
- [Limitations](#limitations)
- [mtime and snapshotting](#mtime-and-snapshotting)
- [kaniko - Build Images In Kubernetes](#kaniko---build-images-in-kubernetes)
- [How does kaniko work?](#how-does-kaniko-work)
- [Known Issues](#known-issues)
- [Demo](#demo)
- [Tutorial](#tutorial)
- [Using kaniko](#using-kaniko)
- [kaniko Build Contexts](#kaniko-build-contexts)
- [Using Private Git Repository](#using-private-git-repository)
- [Running kaniko](#running-kaniko)
- [Running kaniko in a Kubernetes cluster](#running-kaniko-in-a-kubernetes-cluster)
- [Kubernetes secret](#kubernetes-secret)
- [Running kaniko in gVisor](#running-kaniko-in-gvisor)
- [Running kaniko in Google Cloud Build](#running-kaniko-in-google-cloud-build)
- [Running kaniko in Docker](#running-kaniko-in-docker)
- [Caching](#caching)
- [Caching Layers](#caching-layers)
- [Caching Base Images](#caching-base-images)
- [Pushing to Different Registries](#pushing-to-different-registries)
- [Pushing to Docker Hub](#pushing-to-docker-hub)
- [Pushing to Amazon ECR](#pushing-to-amazon-ecr)
- [Additional Flags](#additional-flags)
- [--build-arg](#build-arg)
- [--cache](#cache)
- [--cache-dir](#cache-dir)
- [--cache-repo](#cache-repo)
- [--digest-file](#digest-file)
- [--oci-layout-path](#oci-layout-path)
- [--insecure-registry](#insecure-registry)
- [--skip-tls-verify-registry](#skip-tls-verify-registry)
- [--cleanup](#cleanup)
- [--insecure](#insecure)
- [--insecure-pull](#insecure-pull)
- [--no-push](#no-push)
- [--reproducible](#reproducible)
- [--single-snapshot](#single-snapshot)
- [--skip-tls-verify](#skip-tls-verify)
- [--skip-tls-verify-pull](#skip-tls-verify-pull)
- [--snapshotMode](#snapshotmode)
- [--target](#target)
- [--tarPath](#tarpath)
- [--verbosity](#verbosity)
- [Debug Image](#debug-image)
- [Security](#security)
- [Comparison with Other Tools](#comparison-with-other-tools)
- [Community](#community)
- [Limitations](#limitations)
- [mtime and snapshotting](#mtime-and-snapshotting)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

Expand Down Expand Up @@ -98,6 +103,7 @@ You will need to store your build context in a place that kaniko can access.
Right now, kaniko supports these storage solutions:
- GCS Bucket
- S3 Bucket
- Azure Blob Storage
- Local Directory
- Git Repository

Expand All @@ -123,14 +129,18 @@ When running kaniko, use the `--context` flag with the appropriate prefix to spe

| Source | Prefix | Example |
|---------|---------|---------|
| Local Directory | dir://[path to a directory in the kaniko container] | `dir:///workspace` |
| GCS Bucket | gs://[bucket name]/[path to .tar.gz] | `gs://kaniko-bucket/path/to/context.tar.gz` |
| S3 Bucket | s3://[bucket name]/[path to .tar.gz] | `s3://kaniko-bucket/path/to/context.tar.gz` |
| Git Repository | git://[repository url][#reference] | `git://github.com/acme/myproject.git#refs/heads/mybranch` |
| Local Directory | dir://[path to a directory in the kaniko container] | `dir:///workspace` |
| GCS Bucket | gs://[bucket name]/[path to .tar.gz] | `gs://kaniko-bucket/path/to/context.tar.gz` |
| S3 Bucket | s3://[bucket name]/[path to .tar.gz] | `s3://kaniko-bucket/path/to/context.tar.gz` |
| Azure Blob Storage| https://[account].[azureblobhostsuffix]/[container]/[path to .tar.gz] | `https://myaccount.blob.core.windows.net/container/path/to/context.tar.gz` |
| Git Repository | git://[repository url][#reference] | `git://github.com/acme/myproject.git#refs/heads/mybranch` |

If you don't specify a prefix, kaniko will assume a local directory.
For example, to use a GCS bucket called `kaniko-bucket`, you would pass in `--context=gs://kaniko-bucket/path/to/context.tar.gz`.

### Using Azure Blob Storage
If you are using Azure Blob Storage for context file, you will need to pass [Azure Storage Account Access Key](https://docs.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string?toc=%2fazure%2fstorage%2fblobs%2ftoc.json) as an evironment variable named `AZURE_STORAGE_ACCESS_KEY` through Kubernetes Secrets

### Using Private Git Repository
You can use `Personal Access Tokens` for Build Contexts from Private Repositories from [GitHub](https://blog.github.com/2012-09-21-easier-builds-and-deployments-using-git-over-https-and-oauth/).

Expand Down Expand Up @@ -250,6 +260,17 @@ We can run the kaniko executor image locally in a Docker daemon to build and pus
./run_in_docker.sh <path to Dockerfile> <path to build context> <destination of final image>
```

_NOTE: `run_in_docker.sh` expects a path to a
Dockerfile relative to the absolute path of the build context._

An example run, specifying the Dockerfile in the container directory `/workspace`, the build
context in the local directory `/home/user/kaniko-project`, and a Google Container Registry
as a remote image destination:

```shell
./run_in_docker.sh /workspace/Dockerfile /home/user/kaniko-project gcr.io//<project-id>/<tag>
```

### Caching

#### Caching Layers
Expand Down
4 changes: 2 additions & 2 deletions deploy/Dockerfile_debug
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ FROM gcr.io/cloud-builders/bazel:latest
RUN git clone https://github.com/GoogleContainerTools/distroless.git
WORKDIR /distroless
RUN bazel build //experimental/busybox:busybox_tar
RUN tar -C /distroless/bazel-genfiles/experimental/busybox/ -xf /distroless/bazel-genfiles/experimental/busybox/busybox.tar
RUN tar -C /distroless/bazel-bin/experimental/busybox/ -xf /distroless/bazel-bin/experimental/busybox/busybox.tar

FROM scratch
COPY --from=0 /go/src/github.com/GoogleContainerTools/kaniko/out/* /kaniko/
COPY --from=0 /usr/local/bin/docker-credential-gcr /kaniko/docker-credential-gcr
COPY --from=0 /go/src/github.com/awslabs/amazon-ecr-credential-helper/bin/linux-amd64/docker-credential-ecr-login /kaniko/docker-credential-ecr-login
COPY --from=1 /distroless/bazel-genfiles/experimental/busybox/busybox/ /busybox/
COPY --from=1 /distroless/bazel-bin/experimental/busybox/busybox/ /busybox/
# Declare /busybox as a volume to get it automatically whitelisted
VOLUME /busybox
COPY files/ca-certificates.crt /kaniko/ssl/certs/
Expand Down
23 changes: 23 additions & 0 deletions examples/pod-blobstroage.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: v1
kind: Pod
metadata:
name: kaniko
spec:
containers:
- name: kaniko
image: gcr.io/kaniko-project/executor:latest
args: ["--dockerfile=<path to Dockerfile within the build context>",
"--context=https://myaccount.blob.core.windows.net/container/path/to/context.tar.gz",
"--destination=<registry for image push>"]
...
env:
- name: AZURE_STORAGE_ACCESS_KEY
valueFrom:
secretKeyRef:
name: azure-storage-access-key
key: azure-storage-access-key
...
volumes:
- name: azure-storage-access-key
secret:
secretName: azure-storage-access-key
79 changes: 79 additions & 0 deletions pkg/buildcontext/azureblob.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
/*
Copyright 2018 Google LLC
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package buildcontext

import (
"context"
"errors"
"net/url"
"os"
"path/filepath"
"strings"

"github.com/Azure/azure-storage-blob-go/azblob"
"github.com/GoogleContainerTools/kaniko/pkg/constants"
"github.com/GoogleContainerTools/kaniko/pkg/util"
)

// AzureBlob struct for Azure Blob Storage processing
type AzureBlob struct {
context string
}

// Download context file from given azure blob storage url and unpack it to BuildContextDir
func (b *AzureBlob) UnpackTarFromBuildContext() (string, error) {

// Get Azure_STORAGE_ACCESS_KEY from environment variables
accountKey := os.Getenv("AZURE_STORAGE_ACCESS_KEY")
if len(accountKey) == 0 {
return "", errors.New("AZURE_STORAGE_ACCESS_KEY environment variable is not set")
}

// Get storage accoutname for Azure Blob Storage
u, _ := url.Parse(b.context)
parts := azblob.NewBlobURLParts(*u)
accountName := strings.Split(parts.Host, ".")[0]

// Generate credentail with accountname and accountkey
credential, err := azblob.NewSharedKeyCredential(accountName, accountKey)
if err != nil {
return parts.Host, err
}

// Create directory and target file for downloading the context file
directory := constants.BuildContextDir
tarPath := filepath.Join(directory, constants.ContextTar)
file, err := util.CreateTargetTarfile(tarPath)
if err != nil {
return tarPath, err
}

// Downloading contextfile from Azure Blob Storage
p := azblob.NewPipeline(credential, azblob.PipelineOptions{})
blobURL := azblob.NewBlobURL(*u, p)
ctx := context.Background()

if err := azblob.DownloadBlobToFile(ctx, blobURL, 0, 0, file, azblob.DownloadFromBlobOptions{}); err != nil {
return parts.Host, err
}

if err := util.UnpackCompressedTar(tarPath, directory); err != nil {
return tarPath, err
}
// Remove the tar so it doesn't interfere with subsequent commands
return directory, os.Remove(tarPath)
}
Loading

0 comments on commit 08d0948

Please sign in to comment.