s3 cache client-side support #1294
Conversation
util/buildflags/cache.go
Outdated
| ci.Attrs["secret_access_key"] = v | ||
| } | ||
| } | ||
| if _, ok := ci.Attrs["session_token"]; !ok { |
| @@ -202,15 +206,17 @@ More info about cache exporters and available attributes: https://github.com/mob | |||
| ``` | |||
|
|
|||
| Export build cache to an external cache destination. Supported types are | |||
| `registry`, `local`, `inline` and `gha`. | |||
| `registry`, `local`, `inline`, `gha` and `s3`. | |||
There was a problem hiding this comment.
🎉 Good to have some more docs! I think we probably want to update the README.md as well alongside this? It doesn't seem to have a bullet point under https://github.com/moby/buildkit#export-cache.
There was a problem hiding this comment.
Ah yes looks like it's missing
util/buildflags/cache.go
Outdated
| if ci.Type != "s3" { | ||
| return | ||
| } | ||
| if _, ok := ci.Attrs["access_key_id"]; !ok { |
There was a problem hiding this comment.
This should be done automatically by the AWS SDK. Are you sure we really need to do this?
There was a problem hiding this comment.
I mean we the SDK should be able to configure itself without any manual mapping, For example it's working with AWS IAM Profile. We should look on how to do the same with these env variables.
There was a problem hiding this comment.
Only BuildKit daemon vendor and uses the AWS SDK. The CLI (buildx) does not atm so we rely on aws credentials env vars when available and pass them through as attributes to the daemon. Maybe in a follow-up we could move and expose credentials config retrieval in BuildKit and use it in buildx so we are aligned. WDYT?
There was a problem hiding this comment.
Spent some time to think about it.
Your solution will work only for simple authent (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN). The SDK offers plenty of solutions to authent (example by using AWS_PROFILE), but I do not think there is a simple way to authenticate inside the client and forward creds to the server. Or it do not worth it. So dealing only with simple authent is fine for me, if it's clear in the doc.
crazy-max
force-pushed
the
s3-cache
branch
3 times, most recently
from
056a6bb to
f381a1a
Compare
|
Using the sdk just to load default config brings a lot 😟
|
Signed-off-by: CrazyMax <[email protected]>
follow-up moby/buildkit#2824
fixes docker/build-push-action#647
s3 cache client-side support.
cc @bpaquet
Signed-off-by: CrazyMax [email protected]