Add the option to exclude certain layers from the tarball. #209
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This should be considered a relatively advanced option, but for folks that know what they are doing you can reduce the amount of data that you need to encode in the tarball for the daemon to load it. The ultimate use case of this option will be from `daemon.Write`, which currently uses the `docker load` interface to pull image into the daemon, however, this currently reuploads (and redownloads) the base image on each write in context like `ko`. If we can determine the set of layers that already exist in the daemon we can elide these from the tarball to dramatically improve performance. Related: google#205
| @@ -89,6 +105,15 @@ func Write(tag name.Tag, img v1.Image, wo *WriteOptions, w io.Writer) error { | |||
| // https://www.gnu.org/software/gzip/manual/html_node/Overview.html | |||
| layerFiles[i] = fmt.Sprintf("%s.tar.gz", hex) | |||
|
|
|||
| // We filter late because the lenght of layerFiles must match the diff_ids | |||
| @@ -66,6 +72,16 @@ func Write(tag name.Tag, img v1.Image, wo *WriteOptions, w io.Writer) error { | |||
| return err | |||
| } | |||
|
|
|||
| // Default filter function | |||
There was a problem hiding this comment.
Maybe hoist this out into its own method?
| return true, nil | ||
| } | ||
| if wo != nil { | ||
| if wo.LayerFilter != nil { |
There was a problem hiding this comment.
Collapse this into if wo != nil && wo.LayerFilter != nil { ?
Though I guess that makes it harder to add more options... up to you.
|
@jonjohnsonjr I hear you, but at the same time, I'm not sure of a good way to do this closer to |
|
can you rebase this? |
|
At this point, it's more work to rebase than redo. @jonjohnsonjr are you up for it? |
jonjohnsonjr
added a commit
to jonjohnsonjr/go-containerregistry
that referenced
this pull request
Oct 3, 2019
Continuation of this PR: google#209 This should be considered a relatively advanced option, but for folks that know what they are doing you can reduce the amount of data that you need to encode in the tarball for the daemon to load it. The ultimate use case of this option will be from daemon.Write, which currently uses the docker load interface to pull image into the daemon, however, this currently reuploads (and redownloads) the base image on each write in context like ko. If we can determine the set of layers that already exist in the daemon we can elide these from the tarball to dramatically improve performance. Related: google#205
jonjohnsonjr
added a commit
to jonjohnsonjr/go-containerregistry
that referenced
this pull request
Oct 3, 2019
Continuation of this PR: google#209 This should be considered a relatively advanced option, but for folks that know what they are doing you can reduce the amount of data that you need to encode in the tarball for the daemon to load it. The ultimate use case of this option will be from daemon.Write, which currently uses the docker load interface to pull image into the daemon, however, this currently reuploads (and redownloads) the base image on each write in context like ko. If we can determine the set of layers that already exist in the daemon we can elide these from the tarball to dramatically improve performance. Related: google#205
jonjohnsonjr
added a commit
to jonjohnsonjr/go-containerregistry
that referenced
this pull request
Nov 13, 2019
Continuation of this PR: google#209 This should be considered a relatively advanced option, but for folks that know what they are doing you can reduce the amount of data that you need to encode in the tarball for the daemon to load it. The ultimate use case of this option will be from daemon.Write, which currently uses the docker load interface to pull image into the daemon, however, this currently reuploads (and redownloads) the base image on each write in context like ko. If we can determine the set of layers that already exist in the daemon we can elide these from the tarball to dramatically improve performance. Related: google#205
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This should be considered a relatively advanced option, but for folks that know
what they are doing you can reduce the amount of data that you need to encode in
the tarball for the daemon to load it.
The ultimate use case of this option will be from
daemon.Write, whichcurrently uses the
docker loadinterface to pull image into the daemon,however, this currently reuploads (and redownloads) the base image on each write
in context like
ko. If we can determine the set of layers that already existin the daemon we can elide these from the tarball to dramatically improve
performance.
Related: #205