Add kms decrypt argument for base64 encoded input

[theazureshadow]

The output for aws kms encrypt is a base64-encoded string. The input for aws kms decrypt is a binary string, which is not particularly bash-friendly. See #1100.

It would be useful if there was an additional --ciphertext-base64 argument that could take that same base64 blob from encrypt and decrypt it correctly.

[kyleknap]

Marking as a feature request.

Are you blocked on this? Could you chain the commands with base64 to handle your use case? Like I do in this blog post

[theazureshadow]

I'm not blocked, but it is really non-obvious what's going wrong. We lost a bit of time when trying to validate that we'd encoded our data correctly. Seems like an easy win to be able to specify base64-encoded ciphertext.

[w32-blaster]

I agree with this. There are use cases where it's not trivial to chain commands to get the correct result. It would help in many cases.

[ajlanghorn]

👍 to this.

[iadknet]

👍

[wheresvic]

👍

[rbm1337]

👍

[ghost]

+1

[kid7st]

+1

[sinogermany]

+1

[techypaul]

+1, in the meantime for anyone else looking:

echo AQICAHhb....ZClShBC8dZpCg== | base64 --decode > /tmp/temp.enc
DB_PASS=$(aws kms decrypt --region eu-west-1 --ciphertext-blob fileb:///tmp/temp.enc --query Plaintext --output text | base64 --decode)
sed -i -e "s/<<DB_PASS>>/${DB_PASS}/g" /var/www/mysite/DbSettings.php

[gonzaloserrano]

+1

[jordimirobruix]

+1

[coboshm]

+1

[ryanskinner]

+1

[mlamothe-broadsoft]

+1

[chi42]

+1

[nikhithn]

+1

[scf37]

AFAIK there is no way to call aws kms decrypt from bash with binary string as parameter. So the only way to avoid writing secrets to disk is to use Python and boto.

[lcharkiewicz]

@scf37 there is a way. You need to use process substitution for --ciphertext-blob parameter.
Example:

echo $(aws kms decrypt --ciphertext-blob fileb://<(echo -n "$YOUR_BASE64_CIPHERTEXT" | base64 --decode) --output text --query Plaintext | base64 --decode)

[ASayre]

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

This entry can specifically be found on UserVoice at: https://aws.uservoice.com/forums/598381-aws-command-line-interface/suggestions/33168328-add-kms-decrypt-argument-for-base64-encoded-input

[jamesls]

Based on community feedback, we have decided to return feature requests to GitHub issues.

[jpduckwo]

I'd love to see this feature. Just to reiterate why it's needed....

When you encrypt some data with the kms encrypt command the output is base64 encoded. Base64 encoded encrypted values are widely used. For example storing encrypted environment vars (this is how lamba encryption helpers work).

HOWEVER...

When you call the kms decrypt command the only option is to pass the encrypted data as binary (blob). So all the base64 values have to first be decoded to binary and then decrypted.

AND...

You can't pass the encrypted binary data via stdin to the command - so something like this doesn't work either... base64 --decode encrypted.base64 | aws kms decrypt You have to first convert the file to a binary version and then run the command with --ciphertext-blob

The morale of the story is... PLEASE add --ciphertext-base64 as an option. And while you're at it, it would be good to be able to support stdin too.

🙏😜

[jpduckwo]

Well I'll eat my words... it turns out you can use standard in (on *nix anyway)

Try this:

base64 --decode test.txt.encrypted.base64 | aws kms decrypt --ciphertext-blob fileb:///dev/stdin --output text --query Plaintext | base64 --decode

I hope that helps some of you too!

[mlamothe-broadsoft]

Yes, but the base64 executable still needs to read from a file, which is an hassle and potential security risk to leave lying around.

Just like when encrypting, we need to be able to simply pass the base64 string on the command line which can only be done if that AWS CLI accepts base64 encoded text.

[agrawaltarun]

I am trying to pass the KMS encrypted secret but do not want to create a separate file as do not want to add file handling routines. Surprised this is not implemented though looks like a doable ask.

[mohsinkerai]

+1

[paulieborg]

1 liner with no file (ugly but works)
echo "${KMS_ENCRYPTED_STRING}" | base64 -D | aws kms decrypt --ciphertext-blob fileb:///dev/stdin --output text --query Plaintext | base64 -D
(Props to @jpduckwo )

This is MacOS BTW ... you might need to use base64 -w 0 for Linux - I don't have a instance handy

[kbuchsicf]

I am really supportive of this request. If fileb:///dev/stdin is supported, then that should appear in the documentation. But, it is complicated. I think having --ciphertext-blob-file or --ciphertext-blob-64 would add a lot of clarity. I have a case where the base64 ciphertext is stored in DynamoDB. I need to retrieve that, base64 decode it and save it to a file (or send over a pipe) and then AWS CLI has to turn around and do the base64 encoding. Keeping ciphertexts and plaintexts off the file system is an important security feature.

[rdkls]

--ciphertext-blob-64

I think this the best solution - doesn't break existing implementations, leaving --ciphertext-blob as-is, but adds a sorely needed feature. @kyleknap if I implement will it be endorsed?

[hauntingEcho]

reiterating something from #1043 - this doesn't need to be a separate argument, as : is not a valid character in any base64 variant I'm aware of. It could just be:
--ciphertext-blob proto://path (e.g. fileb://file.bin or file://file.txt)
and
--ciphertext-blob B64String

By my understanding, ciphertext-blob does not accept an argument without a protocol currently (as only fileb is supported), so this would not be a breaking change

[justinmchase]

base64 is itself an accepted protocol in html image tags. It would be easy to just support a base64 protocol on the chiphertext blob, like:

aws kms decrypt --ciphertext-blob base64://${ENCRYPTED_B64_STRING} --output text --query Plaintext

[joguSD]

In CLI v2 we're going with a slightly different approach: #4748

This will add a new parameter / config cli_binary_format which can currently either be set as legacy or base64. In base64 mode blobs passed to the CLI are expected to be a base64 encoded string (no need for a base64://). Having an external configuration for this will allow us to add additional formats as we see fit for the various use cases that might arise for other/future services.

[hauntingEcho]

awesome!
for clarity's sake, a lot of my frustration with this was that encryption and decryption were both binary-in-b64-out (thus requiring conversion between each call). It sounds like base64 will be b64-in-b64-out while legacy will be binary-in-b64 out. Is this correct? Is there intention to enable a binary-in-binary-out option in the future?

either way, it puts us one step closer to echo a | aws kms encrypt --key-id x | aws kms decrypt returning a. Thanks!

[joguSD]

@hauntingEcho legacy input is taken literally output is base64. base64 will be base64 in base64 out. file:// and fileb:// will work in both legacy and base64 mode.

We are looking into expanding this to a raw binary format but it's rather difficult to generalize. For something like kms encrypt or kms decrypt there's only a single binary blob in the output. How it works for operations that have multiple binary fields is something we need to work out.

[tim-finnigan]

There hasn’t been any discussion on this issue for over a couple years. The various approaches mentioned here seem sufficient and as mentioned in this comment we went with a different approach in v2. You can read more about this change here: https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration-changes.html#cliv2-migration-binaryparam. I’m going to close this issue but please let us know if you have any follow up questions.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.