-
Notifications
You must be signed in to change notification settings - Fork 657
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix stream SSE uploads with S3 encrypt type #960
Conversation
You are welcome, I'm implementing the encrypt.NewSSEKMS to try it ;) If I realize that is needed for KMS too, I will send another PR, if you want :) |
That would be awesome :) @jespino |
pkg/encrypt/server-side.go
Outdated
func NewSSEKMS(keyId string) ServerSide { return kms{key: keyId, context: nil} } | ||
|
||
// NewSSEKMSWithContext returns a new server-side-encryption using SSE-KMS and the provided Key Id and context. | ||
func NewSSEKMSWithContext(keyId string, context *string) ServerSide { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func parameter keyId should be keyID
pkg/encrypt/server-side.go
Outdated
@@ -90,6 +95,14 @@ type ServerSide interface { | |||
// Using SSE-S3 the server will encrypt the object with server-managed keys. | |||
func NewSSE() ServerSide { return s3{} } | |||
|
|||
// NewSSEKMS returns a new server-side-encryption using SSE-KMS and the provided Key Id. | |||
func NewSSEKMS(keyId string) ServerSide { return kms{key: keyId, context: nil} } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func parameter keyId should be keyID
pkg/encrypt/server-side.go
Outdated
@@ -30,6 +30,11 @@ const ( | |||
// sseGenericHeader is the AWS SSE header used for SSE-S3 and SSE-KMS. | |||
sseGenericHeader = "X-Amz-Server-Side-Encryption" | |||
|
|||
// sseKmsKeyId is the AWS SSE-KMS key id. | |||
sseKmsKeyId = sseGenericHeader + "-Aws-Kms-Key-Id" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const sseKmsKeyId should be sseKmsKeyID
I have created what I think is needed, but I don't know too much about KMS, so I don't know if the context as an optional string is enough. |
24bcabc
to
ef97e05
Compare
pkg/encrypt/server-side.go
Outdated
func NewSSEKMS(keyID string) ServerSide { return kms{key: keyID, context: nil} } | ||
|
||
// NewSSEKMSWithContext returns a new server-side-encryption using SSE-KMS and the provided Key Id and context. | ||
func NewSSEKMSWithContext(keyID string, context *string) ServerSide { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since::
The encryption context can be any value that you want, provided that the header adheres to the Base64-encoded JSON format.
I think we can follow this approach:
NewSSEKMS(keyID string, context interface{}) (ServerSide, error) {
// if context is nil -> no context
// if context implements json.Marshaler -> marshal
// json.Marshal(context)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this approach and then we don't need the contextPresent
proposed in the comment below. I'll do this changes tomorrow.
pkg/encrypt/server-side.go
Outdated
@@ -90,6 +95,14 @@ type ServerSide interface { | |||
// Using SSE-S3 the server will encrypt the object with server-managed keys. | |||
func NewSSE() ServerSide { return s3{} } | |||
|
|||
// NewSSEKMS returns a new server-side-encryption using SSE-KMS and the provided Key Id. | |||
func NewSSEKMS(keyID string) ServerSide { return kms{key: keyID, context: nil} } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we don't need two functions right now - removing a function breaks API while we can always add if requested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, perfect, I'll leave one only.
pkg/encrypt/server-side.go
Outdated
|
||
type kms struct { | ||
key string | ||
context *string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use a bool var - like contextPresent
instead of a *string
and nil checks?
pkg/encrypt/server-side.go
Outdated
} | ||
if serializedContext, err := json.Marshal(context); err != nil { | ||
return nil, err | ||
} else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
6216efc
to
63401ff
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
When you try to upload files stream based with encryption activated, minio-go
try to upload the parts using the ServerSideEncryption header, but it isn't
supported for uploadPart with encrypt type S3, only is needed for customer
keys. For S3 encryption we simply add the header at the creation, and in the
following uploadPart we don't pass the header.
I'm not sure, but maybe we need to exclude the encrypt KMS type too (I'm not
sure).
Here is an example of it failing before the patch, and working after it.