Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying binary path in image #1128

Closed
wants to merge 3 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference/ko_apply.md
Original file line number Diff line number Diff line change
@@ -47,6 +47,7 @@ ko apply -f FILENAME [flags]
```
--bare Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
-B, --base-import-paths Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
--binary string Set to override binary path in image.
--debug Include Delve debugger into image and wrap around ko-app. This debugger will listen to port 40000.
--disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
-f, --filename strings Filename, directory, or URL to files to use to create the resource
1 change: 1 addition & 0 deletions docs/reference/ko_build.md
Original file line number Diff line number Diff line change
@@ -44,6 +44,7 @@ ko build IMPORTPATH... [flags]
```
--bare Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
-B, --base-import-paths Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
--binary string Set to override binary path in image.
--debug Include Delve debugger into image and wrap around ko-app. This debugger will listen to port 40000.
--disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
-h, --help help for build
1 change: 1 addition & 0 deletions docs/reference/ko_create.md
Original file line number Diff line number Diff line change
@@ -47,6 +47,7 @@ ko create -f FILENAME [flags]
```
--bare Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
-B, --base-import-paths Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
--binary string Set to override binary path in image.
--debug Include Delve debugger into image and wrap around ko-app. This debugger will listen to port 40000.
--disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
-f, --filename strings Filename, directory, or URL to files to use to create the resource
1 change: 1 addition & 0 deletions docs/reference/ko_resolve.md
Original file line number Diff line number Diff line change
@@ -40,6 +40,7 @@ ko resolve -f FILENAME [flags]
```
--bare Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
-B, --base-import-paths Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
--binary string Set to override binary path in image.
--debug Include Delve debugger into image and wrap around ko-app. This debugger will listen to port 40000.
--disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
-f, --filename strings Filename, directory, or URL to files to use to create the resource
1 change: 1 addition & 0 deletions docs/reference/ko_run.md
Original file line number Diff line number Diff line change
@@ -32,6 +32,7 @@ ko run IMPORTPATH [flags]
```
--bare Whether to just use KO_DOCKER_REPO without additional context (may not work properly with --tags).
-B, --base-import-paths Whether to use the base path without MD5 hash after KO_DOCKER_REPO (may not work properly with --tags).
--binary string Set to override binary path in image.
--debug Include Delve debugger into image and wrap around ko-app. This debugger will listen to port 40000.
--disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container.
-h, --help help for run
4 changes: 3 additions & 1 deletion pkg/build/config.go
Original file line number Diff line number Diff line change
@@ -79,14 +79,16 @@ type Config struct {
// Env allows setting environment variables for `go build`
Env []string `yaml:",omitempty"`

// Binary allows overriding the output binary name (in the image)
Binary string `yaml:",omitempty"`

// Other GoReleaser fields that are not supported or do not make sense
// in the context of ko, for reference or for future use:
// Goos []string `yaml:",omitempty"`
// Goarch []string `yaml:",omitempty"`
// Goarm []string `yaml:",omitempty"`
// Gomips []string `yaml:",omitempty"`
// Targets []string `yaml:",omitempty"`
// Binary string `yaml:",omitempty"`
// Lang string `yaml:",omitempty"`
// Asmflags StringArray `yaml:",omitempty"`
// Gcflags StringArray `yaml:",omitempty"`
73 changes: 52 additions & 21 deletions pkg/build/gobuild.go
Original file line number Diff line number Diff line change
@@ -92,6 +92,7 @@ type gobuild struct {
build builder
sbom sbomber
sbomDir string
binaryPath string
disableOptimizations bool
trimpath bool
buildConfigs map[string]Config
@@ -118,6 +119,7 @@ type gobuildOpener struct {
build builder
sbom sbomber
sbomDir string
binaryPath string
disableOptimizations bool
trimpath bool
buildConfigs map[string]Config
@@ -150,6 +152,7 @@ func (gbo *gobuildOpener) Open() (Interface, error) {
build: gbo.build,
sbom: gbo.sbom,
sbomDir: gbo.sbomDir,
binaryPath: gbo.binaryPath,
disableOptimizations: gbo.disableOptimizations,
trimpath: gbo.trimpath,
buildConfigs: gbo.buildConfigs,
@@ -587,25 +590,39 @@ func tarBinary(name, binary string, platform *v1.Platform, opts *layerOptions) (
// For Windows, the layer must contain a Hives/ directory, and the root
// of the actual filesystem goes in a Files/ directory.
// For Linux, the binary goes into /ko-app/
dirs := []string{"ko-app"}
appDir := filepath.Dir(name)
dirs := []string{appDir}
if platform.OS == "windows" {
dirs = []string{
"Hives",
"Files",
"Files/ko-app",
"Files/" + appDir,
}
name = "Files" + name
}
for _, dir := range dirs {
if err := tw.WriteHeader(&tar.Header{
Name: dir,
Typeflag: tar.TypeDir,
// Use a fixed Mode, so that this isn't sensitive to the directory and umask
// under which it was created. Additionally, windows can only set 0222,
// 0444, or 0666, none of which are executable.
Mode: 0555,
}); err != nil {
return nil, fmt.Errorf("writing dir %q to tar: %w", dir, err)
// Create all parent directories also
var parents []string
current := dir
for {
parents = append(parents, current)
current = filepath.Dir(current)
if current == "/" {
break
}
}

for i := len(parents) - 1; i >= 0; i-- {
parent := parents[i]
if err := tw.WriteHeader(&tar.Header{
Name: parent,
Typeflag: tar.TypeDir,
// Use a fixed Mode, so that this isn't sensitive to the directory and umask
// under which it was created. Additionally, windows can only set 0222,
// 0444, or 0666, none of which are executable.
Mode: 0555,
}); err != nil {
return nil, fmt.Errorf("writing dir %q to tar: %w", parent, err)
}
}
}

@@ -916,6 +933,10 @@ func (g *gobuild) configForImportPath(ip string) Config {
config.Flags = append(config.Flags, "-gcflags", "all=-N -l")
}

if g.binaryPath != "" {
config.Binary = g.binaryPath
}

if config.ID != "" {
log.Printf("Using build config %s for %s", config.ID, ip)
}
@@ -927,6 +948,14 @@ func (g gobuild) useDebugging(platform v1.Platform) bool {
return g.debug && doesPlatformSupportDebugging(platform)
}

// pathToWindows converts a unix-style path to a windows-style path.
// For example, /apps/foo => C:\apps\foo
func pathToWindows(s string) string {
pathComponents := []string{"C:"}
pathComponents = append(pathComponents, strings.Split(s, "/")...)
return strings.Join(pathComponents, `\`)
}

func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, platform *v1.Platform) (oci.SignedImage, error) {
if err := g.semaphore.Acquire(ctx, 1); err != nil {
return nil, err
@@ -1043,9 +1072,12 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl
},
})

appDir := "/ko-app"
appFileName := appFilename(ref.Path())
appPath := path.Join(appDir, appFileName)
appPath := config.Binary
if appPath == "" {
appPath = path.Join("/ko-app", appFilename(ref.Path()))
}
appDir := path.Dir(appPath)
appFileName := path.Base(appPath)

var lo layerOptions
lo.linuxCapabilities, err = caps.NewFileCaps(config.LinuxCapabilities...)
@@ -1136,16 +1168,15 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl
cfg.Config.Entrypoint = []string{appPath}
cfg.Config.Cmd = nil
if platform.OS == "windows" {
appPath := `C:\ko-app\` + appFileName
if g.debug {
cfg.Config.Entrypoint = append([]string{"C:\\" + delvePath}, delveArgs...)
cfg.Config.Entrypoint = append(cfg.Config.Entrypoint, appPath)
cfg.Config.Entrypoint = append([]string{pathToWindows(delvePath)}, delveArgs...)
cfg.Config.Entrypoint = append(cfg.Config.Entrypoint, pathToWindows(appPath))
} else {
cfg.Config.Entrypoint = []string{appPath}
cfg.Config.Entrypoint = []string{pathToWindows(appPath)}
}

updatePath(cfg, `C:\ko-app`)
cfg.Config.Env = append(cfg.Config.Env, `KO_DATA_PATH=C:\var\run\ko`)
updatePath(cfg, pathToWindows(filepath.Dir(appPath)))
cfg.Config.Env = append(cfg.Config.Env, `KO_DATA_PATH=`+pathToWindows(kodataRoot))
} else {
if g.useDebugging(*platform) {
cfg.Config.Entrypoint = append([]string{delvePath}, delveArgs...)
125 changes: 111 additions & 14 deletions pkg/build/gobuild_test.go
Original file line number Diff line number Diff line change
@@ -496,6 +496,16 @@
Flags: FlagArray{"-gcflags", "all=-N -l"},
},
},
{
description: "override binary path",
options: []Option{
WithBaseImages(nilGetBase),
WithBinaryPath("/mydir/myprogram"),
},
expectConfig: Config{
Binary: "/mydir/myprogram",
},
},
}
for _, test := range tests {
t.Run(test.description, func(t *testing.T) {
@@ -640,9 +650,29 @@
})
}

func validateImage(t *testing.T, img oci.SignedImage, baseLayers int64, creationTime v1.Time, checkAnnotations bool, expectSBOM bool) {
// validationOptions allows for passing additional options to validateImage.
type validationOptions struct {
// Entrypoint contains the expected image entrypoint.
// If not set, defaults to /ko-app/test.
Entrypoint string
// AppLayerEntries is the expected names of the entries in the app layer.
AppLayerEntries []string
// DisableSBOM is true if we have turned off SBOM generation.
DisableSBOM bool
// IgnoreAnnotations can be set to true to ignore the annotations check.
IgnoreAnnotations bool
}

func validateImage(t *testing.T, img oci.SignedImage, baseLayers int64, creationTime v1.Time, opt validationOptions) {

Check failure on line 666 in pkg/build/gobuild_test.go

GitHub Actions / lint

`validateImage` - `baseLayers` always receives `baseLayers` (`3`) (unparam)
t.Helper()

if opt.Entrypoint == "" {
opt.Entrypoint = "/ko-app/test"
}
if opt.AppLayerEntries == nil {
opt.AppLayerEntries = []string{"/ko-app,/ko-app/test"}
}

ls, err := img.Layers()
if err != nil {
t.Fatalf("Layers() = %v", err)
@@ -657,22 +687,44 @@
})

t.Run("check app layer contents", func(t *testing.T) {
dataLayer := ls[baseLayers]
appLayer := ls[baseLayers+1]

if _, err := dataLayer.Digest(); err != nil {
if _, err := appLayer.Digest(); err != nil {
t.Errorf("Digest() = %v", err)
}
// We don't check the data layer here because it includes a symlink of refs and
// We don't check the app layer hash here because it includes a symlink of refs and
// will produce a distinct hash each time we commit something.

r, err := dataLayer.Uncompressed()
r, err := appLayer.Uncompressed()
if err != nil {
t.Errorf("Uncompressed() = %v", err)
}
defer r.Close()
var entries []tar.Header
tr := tar.NewReader(r)
if _, err := tr.Next(); errors.Is(err, io.EOF) {
t.Errorf("Layer contained no files")
for {
header, err := tr.Next()
if errors.Is(err, io.EOF) {
break
} else if err != nil {
t.Errorf("Next() = %v", err)
continue
}
entries = append(entries, *header)
if _, err := io.Copy(io.Discard, tr); err != nil {
t.Errorf("Copy() = %v", err)
}
}
if len(entries) == 0 {
t.Error("Didn't find expected file in tarball")
}
var names []string
for _, entry := range entries {
names = append(names, entry.Name)
}
wantNames := strings.Join(opt.AppLayerEntries, ",")
if got, want := wantNames, strings.Join(names, ","); got != want {
t.Errorf("entry names = %v, want %v", got, want)
}
})

@@ -722,7 +774,7 @@
t.Errorf("len(entrypoint) = %v, want %v", got, want)
}

if got, want := entrypoint[0], "/ko-app/test"; got != want {
if got, want := entrypoint[0], opt.Entrypoint; got != want {
t.Errorf("entrypoint = %v, want %v", got, want)
}
})
@@ -756,7 +808,7 @@
pathValue := strings.TrimPrefix(envVar, "PATH=")
pathEntries := strings.Split(pathValue, ":")
for _, pathEntry := range pathEntries {
if pathEntry == "/ko-app" {
if pathEntry == filepath.Dir(opt.Entrypoint) {
found = true
}
}
@@ -780,7 +832,7 @@
})

t.Run("check annotations", func(t *testing.T) {
if !checkAnnotations {
if opt.IgnoreAnnotations {
t.Skip("skipping annotations check")
}
mf, err := img.Manifest()
@@ -797,7 +849,7 @@
}
})

if expectSBOM {
if !opt.DisableSBOM {
t.Run("checking for SBOM", func(t *testing.T) {
f, err := img.Attachment("sbom")
if err != nil {
@@ -860,7 +912,7 @@
t.Fatalf("Build() not a SignedImage: %T", result)
}

validateImage(t, img, baseLayers, creationTime, true, true)
validateImage(t, img, baseLayers, creationTime, validationOptions{})

// Check that rebuilding the image again results in the same image digest.
t.Run("check determinism", func(t *testing.T) {
@@ -1068,7 +1120,52 @@
t.Fatalf("Build() not a SignedImage: %T", result)
}

validateImage(t, img, baseLayers, creationTime, true, false)
validateImage(t, img, baseLayers, creationTime, validationOptions{DisableSBOM: true})
}

func TestGoBuildWithBinary(t *testing.T) {
baseLayers := int64(3)
base, err := random.Image(1024, baseLayers)
if err != nil {
t.Fatalf("random.Image() = %v", err)
}
importpath := "github.com/google/ko"

creationTime := v1.Time{Time: time.Unix(5000, 0)}
ng, err := NewGo(
context.Background(),
"",
WithCreationTime(creationTime),
WithBaseImages(func(context.Context, string) (name.Reference, Result, error) { return baseRef, base, nil }),
withBuilder(writeTempFile),
withSBOMber(fauxSBOM),
WithLabel("foo", "bar"),
WithLabel("hello", "world"),
WithPlatforms("all"),
WithConfig(map[string]Config{
"github.com/google/ko/test": {
Binary: "/custom/path/to/binary",
},
}),
)
if err != nil {
t.Fatalf("NewGo() = %v", err)
}

result, err := ng.Build(context.Background(), StrictScheme+filepath.Join(importpath, "test"))
if err != nil {
t.Fatalf("Build() = %v", err)
}

img, ok := result.(oci.SignedImage)
if !ok {
t.Fatalf("Build() not a SignedImage: %T", result)
}

validateImage(t, img, baseLayers, creationTime, validationOptions{
Entrypoint: "/custom/path/to/binary",
AppLayerEntries: []string{"/custom", "/custom/path", "/custom/path/to", "/custom/path/to/binary"},
})
}

func TestGoBuildIndex(t *testing.T) {
@@ -1114,7 +1211,7 @@
if err != nil {
t.Fatalf("idx.Image(%s) = %v", desc.Digest, err)
}
validateImage(t, img, baseLayers, creationTime, false, true)
validateImage(t, img, baseLayers, creationTime, validationOptions{IgnoreAnnotations: true})
}

if want, got := images, int64(len(im.Manifests)); want != got {
9 changes: 9 additions & 0 deletions pkg/build/options.go
Original file line number Diff line number Diff line change
@@ -191,3 +191,12 @@ func WithDebugger() Option {
return nil
}
}

// WithBinaryPath is a functional option for overriding the path
// to the executable in the output image.
func WithBinaryPath(binaryPath string) Option {
return func(gbo *gobuildOpener) error {
gbo.binaryPath = binaryPath
return nil
}
}
5 changes: 5 additions & 0 deletions pkg/commands/options/build.go
Original file line number Diff line number Diff line change
@@ -78,6 +78,9 @@ type BuildOptions struct {
// `AddBuildOptions()` defaults this field to `true`.
Trimpath bool

// BinaryPath overrides the default path for the binary in the output image.
BinaryPath string

// BuildConfigs stores the per-image build config from `.ko.yaml`.
BuildConfigs map[string]build.Config
}
@@ -93,6 +96,8 @@ func AddBuildOptions(cmd *cobra.Command, bo *BuildOptions) {
"Path to file where the SBOM will be written.")
cmd.Flags().StringSliceVar(&bo.Platforms, "platform", []string{},
"Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*")
cmd.Flags().StringVar(&bo.BinaryPath, "binary", "",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we rename this to --binary-path instead?

"Set to override binary path in image.")
cmd.Flags().StringSliceVar(&bo.Labels, "image-label", []string{},
"Which labels (key=value) to add to the image.")
cmd.Flags().BoolVar(&bo.Debug, "debug", bo.Debug,
4 changes: 4 additions & 0 deletions pkg/commands/resolver.go
Original file line number Diff line number Diff line change
@@ -128,6 +128,10 @@ func gobuildOptions(bo *options.BuildOptions) ([]build.Option, error) {
opts = append(opts, build.WithSBOMDir(bo.SBOMDir))
}

if bo.BinaryPath != "" {
opts = append(opts, build.WithBinaryPath(bo.BinaryPath))
}

return opts, nil
}