Transit encrypt batch does not honor key_version

[vsevel]

Describe the bug
I create a transit key for encryption, rotate it, and attempt to encrypt a batch with key_version=1.
I expect to receive a cipher vault:v1. instead I get a v2.

To Reproduce
Steps to reproduce the behavior:

docker run --cap-add=IPC_LOCK -d -p 8200:8200 --rm --name=dev-vault vault
export VAULT_TOKEN=s.HweVp01G5DU1zbXWlteoCI9G
vault secrets enable transit
vault write -f transit/keys/my-key
vault write -f transit/keys/my-key/rotate
vault read transit/keys/my-key

Key                       Value
---                       -----
allow_plaintext_backup    false
deletion_allowed          false
derived                   false
exportable                false
keys                      map[1:1603534336 2:1603534432]
latest_version            2
min_available_version     0
min_decryption_version    1
min_encryption_version    0
name                      my-key
supports_decryption       true
supports_derivation       true
supports_encryption       true
supports_signing          false
type                      aes256-gcm96

# create payload.json:

{
    "key_version": 1,
    "batch_input": [{"plaintext": "dGhlIHF1aWNrIGJyb3duIGZveA=="}]
}

curl --header "X-Vault-Token: s.HweVp01G5DU1zbXWlteoCI9G" --request POST --data @payload.json http://127.0.0.1:8200/v1/transit/encrypt/my-key

# should return "ciphertext":"vault:v1, instead it returns:
# {"request_id":"718cf97e-51a4-a86b-e409-34b1a6148001","lease_id":"","renewable":false,"lease_duration":0,"data":{"batch_results":[{"ciphertext":"vault:v2:gEhyXQ+HOy4glO5AeIkokX7osbztINx1+OJOzIPLEHyzAAim7pQ8alX+kKcUQQs=","key_version":2}]},"wrap_info":null,"warnings":null,"auth":null}

If I try to move the key_version back into the batch input item:

{
    "batch_input": [
        {
            "key_version": 1,
            "plaintext": "dGhlIHF1aWNrIGJyb3duIGZveA=="
        }
    ]
}

Then I get:

curl --header "X-Vault-Token: s.p8DoelaVTCc9OFlxZmBswBxD" --request POST --data @payload.json http://127.0.0.1:8200/v1/transit/encrypt/my-key
{"errors":["1 error occurred:\n\t* failed to parse batch input: 1 error(s) decoding:\n\n* '[0].key_version' expected type 'int', got unconvertible type 'json.Number'\n\n"]}

Expected behavior
Should return a v1 cipher.

Environment:

• Vault Server Version (retrieve with vault status): 1.5.4
• Vault CLI Version (retrieve with vault version): 1.3.0
• Server Operating System/Architecture: macos

Additional context
was working in 1.2.2.
also key_version used to be supported in the batch_input. now it is one level up, meaning that there could be only one version for the entire batch. is that correct?

in the previous instructions, replace payload with:

{    
    "batch_input": [{"key_version": 1, "plaintext": "dGhlIHF1aWNrIGJyb3duIGZveA=="}]
}

and execute curl --header "X-Vault-Token: s.NKjYBQxjieOVuIviBsKqMi6r" --request POST --data @payload.json http://127.0.0.1:8200/v1/transit/encrypt/my-key.
then you get as expected a v1 cipher:

{"request_id":"3957222f-a237-e28c-20e9-46de0ec2ef15","lease_id":"","renewable":false,"lease_duration":0,"data":{"batch_results":[{"ciphertext":"vault:v1:4NFGX6ej5pDaD0lB9blzfVw7Th+yBWtYkyjmB9i3D72WMagYM4nlyppBzaNb85k="}]},"wrap_info":null,"warnings":null,"auth":null}

[irvingd]

We're running in to the same problem. We have a key with three versions. Regardless of the version, the encryption is always done with the latest key.

However, if using the Vault UI, it does the right thing (i.e. it uses the correct key version)

[vsevel]

@irvingd

However, if using the Vault UI, it does the right thing (i.e. it uses the correct key version)

that is because the ui does not use a batch input. here is what it does:

curl 'http://localhost:8200/v1/transit/encrypt/my-key' --header "X-Vault-Token: $VAULT_TOKEN" --data-binary '{"plaintext":"Y291Y291","key_version":1}'           
      
{"request_id":"1c48893c-ccbd-7287-91a5-040f1b29336b","lease_id":"","renewable":false,"lease_duration":0,"data":{"ciphertext":"vault:v1:wQ+MUSqP4sWlAtAeVYzMnAwnpas5U/77n/xuZ8TadxpVAg==","key_version":1},"wrap_info":null,"warnings":null,"auth":null}

and this should work too:

curl 'http://localhost:8200/v1/transit/encrypt/my-key' --header "X-Vault-Token: $VAULT_TOKEN" --data-binary '{"batch_input":[{"plaintext":"Y291Y291","key_version":1}]}' 

{"errors":["1 error occurred:\n\t* failed to parse batch input: 1 error(s) decoding:\n\n* '[0].key_version' expected type 'int', got unconvertible type 'json.Number'\n\n"]}

[rajanadar]

One of the users of VaultSharp (a .NET Client for Vault) hit the same issue of key_version type casting error, when used inside a batch item. This comment from me has some analysis on the cause of the casting exception in Vault.

rajanadar/VaultSharp#184 (comment)

The only workaround for clients right now, is to loop through manually and encrypt 1 item at a time (what the vault UI does to not encounter this issue). Batch encrypt won't work if you need a non-latest key version.

Any updates, Vault team?

FYI @bentdan

[sgmiller]

Thanks everyone. We've merged the fix for the next major release as well as the next 1.7 point release.