Cannot use AZCOPY_OAUTH_TOKEN_INFO to auth in 10.21.0 version

[RobinLin666]

Which version of the AzCopy was used?

10.21.0

Note: The version is visible when running AzCopy without any argument

Which platform are you using? (ex: Windows, Mac, Linux)

Linux

What command did you run?

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

In Fabric notebook, I want to use azcopy via 'AZCOPY_OAUTH_TOKEN_INFO'. 10.18.1 works, but 10.21.0 breaks.

import os
from trident_token_library_wrapper import PyTridentTokenLibrary
aad_token = PyTridentTokenLibrary.get_access_token("storage")
import jwt, json
token = aad_token
token_json = jwt.decode(token, algorithms='RS256', options={"verify_signature": False})
# token = token[1:]
token_info = {
    "access_token": token,
    "token_type": "Bearer",
    "not_before": token_json.get("nbf", ""),
    "expires_on": token_json.get("exp", ""),
    "expires_in": str(
        int(float(token_json.get("exp", "0")) - float(token_json.get("nbf", "0")))),
    "resource": "https://storage.azure.com",
    "_token_refresh_source": "tokenstore"
}
os.environ.setdefault("AZCOPY_OAUTH_TOKEN_INFO", json.dumps(token_info))

!azcopy list 'https://xxx.blob.core.windows.net/'

What problem was encountered?

INFO: AZCOPY_OAUTH_TOKEN_INFO is set.
INFO: Autologin not specified.
INFO: Authenticating to source using Azure AD

failed to traverse container: cannot list files due to reason no cached token found in Token Store Mode(SE), failed to find token from gnome keyring, Not implemented

How can we reproduce the problem in the simplest way?

Have you found a mitigation/solution?

[siminsavani-msft]

Thank you for reporting this! We are taking a look at this.

[siminsavani-msft]

Hi @RobinLin666 ! AZCOPY_OAUTH_TOKEN_INFO is for internal usage only, see the comment from the AzCopy codebase below.

azure-storage-azcopy/common/environment.go

Lines 309 to 312 in 7f3ebae

	// OAuthTokenInfo is only used for internal integration.
	func (EnvironmentVariable) OAuthTokenInfo() EnvironmentVariable {
	return EnvironmentVariable{Name: "AZCOPY_OAUTH_TOKEN_INFO"}
	}

With this being said, was this working for you in previous versions?

Additionally, we recommend all our customers follow this documentation (https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-authorize-azure-active-directory) in regard to authorizing with AzCopy.

[RobinLin666]

Hi @siminsavani-msft,
Thank you very much for your reply!
I have read the doc but I didn't find a way to authorizing with AAD Token.

This is how I use it in v10.18.1.

import os
aad_token = mssparkutils.credential.getToken("storage")
import jwt, json
token = aad_token
token_json = jwt.decode(token, algorithms='RS256', options={"verify_signature": False})
print(token_json)
# token = token[1:]
token_info = {
    "access_token": token,
    "token_type": "Bearer",
    "not_before": token_json.get("nbf", ""),
    "expires_on": token_json.get("exp", ""),
    "expires_in": str(
        int(float(token_json.get("exp", "0")) - float(token_json.get("nbf", "0")))),
    "resource": "https://storage.azure.com",
    "_token_refresh_source": "tokenstore"
}
os.environ.setdefault("AZCOPY_OAUTH_TOKEN_INFO", json.dumps(token_info))
 
# run bash shell
!azcopy list https://xxx.blob.core.windows.net/xxx

[siminsavani-msft]

@RobinLin666 AAD has been renamed to Entra ID. The documentation contains the latest way to authorize with AzCopy. I would recommend checking out the managed identity or service principal sections of the doc.

I'll continue investigating the AZCOPY_OAUTH_TOKEN_INFO and update this thread accordingly. In the meantime, I would recommend updating your script to use the suggested methods.

[RobinLin666]

Thank you very much, I will check the doc.

[siminsavani-msft]

Just an update, I was able to repro your issue and have marked this as a bug.

We will continue working on this issue and update the thread with a fix in the next few upcoming releases.

[gapra-msft]

The 10.22.0 version was just released and the above PR was included in the release. Closing this issue as a result.

Please feel free to open a new issue if you are experiencing any issues with latest AzCopy and we are happy to help.