v3.9.2 fails to decrypt with Azure KeyVault

[SlashDashAndCash]

Since v3.9.2 I'm unable to decrypt my data.

./sops-v3.9.2.linux.amd64 -d secrets.sops.yaml
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  https://XXXXXXXXXXX.vault.azure.net/keys/sops/0123456789abcdef: FAILED
    - | failed to decrypt sops data key with Azure Key Vault key
      | 'https://XXXXXXXXXXX.vault.azure.net/keys/sops/0123456789abcdef':
      | DefaultAzureCredential: failed to acquire a token.
      | Attempted credentials:
      |         EnvironmentCredential: missing environment variable
      | AZURE_TENANT_ID
      |         WorkloadIdentityCredential: no client ID specified. Check
      | pod configuration or set ClientID in the options
      |         ManagedIdentityCredential: managed identity timed out. See
      | https://aka.ms/azsdk/go/identity/troubleshoot#dac for more
      | information
      |         AzureCLICredential isn't configured to acquire tokens for
      | tenant "ab1234cd-a1b2-c3d4-e5f6-ab1234cda1b2". To enable
      | acquiring tokens for this tenant add it to the
      | AdditionallyAllowedTenants on the credential options, or add
      | "*" to allow acquiring tokens for any tenant

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

v3.9.1 still works as expected.

./sops-v3.9.1.linux.amd64 -d secrets.sops.yaml
secrets:
...

[ThomasShih]

+1, we are also experiencing this. Will downgrade versions for now.

[svenbuerger]

Had same issue, setting:
export AZURE_ADDITIONALLY_ALLOWED_TENANTS=*
helped as the error message suggests.

[felixfontein]

Considering the error message and that setting AZURE_ADDITIONALLY_ALLOWED_TENANTS helped @svenbuerger, my guess is that this is related to updating the Azure SDK. The only code change on SOPS' side that affects Azure is #1652, which should not have this effect.

[Undermyspell]

i tested locally with the current versions from sops 3.9.2

github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys v1.3.0 which uses
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0.

When i try to decrypt a value using this versions i get the same error from the azcore package

From sops 3.9.1 to 3.9.2 both packages were upgraded to the latest version, so i tested the 3.9.1 versions as well

Downgrading to
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys v1.1.0 and
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.13.0

does not raise any error and works as expected

Note: The github.com/Azure/azure-sdk-for-go/sdk/azidentity does not make any problems and the latest v1.8.0 can be used in both cases

Below is a simple code ro reproduce

package main

import (
	"context"

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys"
	"github.com/sirupsen/logrus"
)

func main() {

	cred, _ := azidentity.NewDefaultAzureCredential(nil)

	c, err := azkeys.NewClient("[YOUR_KV_URL]", cred, nil)
	if err != nil {
		logrus.Fatal(err.Error())
	}

	resp, err := c.Decrypt(context.Background(), "[YOUR_KEY]", "[YOUR_KEY_VERSION]", azkeys.KeyOperationParameters{
		Algorithm: to.Ptr(azkeys.EncryptionAlgorithmRSAOAEP256),
		Value:     [YOUR_VALUE],   // []byte{0, 1} does suffice to validate the test
        }, nil)

	if err != nil {
		logrus.Fatal(err.Error())
	}

	logrus.Info(resp)
}

[cippaciong]

Someone opened this issue in https://github.com/Azure/azure-sdk-for-go which seems related to what we see here: Azure/azure-sdk-for-go#23851

[Pluggi]

Azure/azure-sdk-for-go#23851 (comment) was marked as resolved yesterday.

Has anyone tried to see if it fixes our issue?

[felixfontein]

#1734 bumps the version of github.com/Azure/azure-sdk-for-go/sdk/azidentity to 1.8.1, if that helps testing.

[Pluggi]

Yes, it does!
I just compiled it in a temp directory and got it to work without any changes 🚀

❯ sops --version
sops 3.9.3 (latest)

❯ sops file.yaml
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  arn:aws:kms:REDACTED:key/REDACTED: FAILED
    - | failed to decrypt sops data key with AWS KMS: operation
      | error KMS: Decrypt, get identity: get credentials: failed to
      | refresh cached credentials, refresh cached SSO token failed,
      | unable to refresh SSO token, operation error SSO OIDC:
      | CreateToken, https response error StatusCode: 400,
      | RequestID: REDACTED,
      | InvalidGrantException:

  https://REDACTED.vault.azure.net/keys/sops/REDACTED: FAILED
    - | failed to decrypt sops data key with Azure Key Vault key
      | 'https://REDACTED.vault.azure.net/keys/sops/REDACTED':
      | DefaultAzureCredential: failed to acquire a token.
      | Attempted credentials:
      | 	EnvironmentCredential: missing environment variable
      | AZURE_TENANT_ID
      | 	WorkloadIdentityCredential: no client ID specified. Check
      | pod configuration or set ClientID in the options
      | 	ManagedIdentityCredential: managed identity timed out. See
      | https://aka.ms/azsdk/go/identity/troubleshoot#dac for more
      | information
      | 	AzureCLICredential isn't configured to acquire tokens for
      | tenant "REDACTED". To enable
      | acquiring tokens for this tenant add it to the
      | AdditionallyAllowedTenants on the credential options, or add
      | "*" to allow acquiring tokens for any tenant

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

❯ /tmp/tmp.ZdmzwkaQLR/sops file.yaml
File has not changed, exiting.

[felixfontein]

Great to hear!

[jarpoole]

@felixfontein any chance we could get a release when you have a moment?

[felixfontein]

I'm currently waiting for #1733 to get reviewed by another maintainer, then I'll start working on a new bugfix release.