Detection of Azure hierarchical namespaces fails with ResourceNotFoundError

[M0dEx]

In the latest 0.19.0 version, all Azure operations fail with our Azure Blob Storage accounts with the ResourceNotFoundError.

The traceback leads to the get_account_information method used for HNS detection:

Traceback

Traceback (most recent call last):
File ".../PyCharm Professional Edition.app/Contents/plugins/python/helpers-pro/pydevd_asyncio/pydevd_asyncio_utils.py", line 117, in _exec_async_code
  result = func()
           ^^^^^^
File "<input>", line 1, in <module>
File ".../lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 94, in wrapper_use_tracer
  return func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/azure/storage/blob/_blob_service_client.py", line 245, in get_account_information
  process_storage_error(error)
File ".../lib/python3.11/site-packages/azure/storage/blob/_shared/response_handlers.py", line 186, in process_storage_error
  exec("raise error from None")   # pylint: disable=exec-used # nosec
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 1, in <module>
File ".../lib/python3.11/site-packages/azure/storage/blob/_blob_service_client.py", line 243, in get_account_information
  return self._client.service.get_account_info(cls=return_response_headers, **kwargs) # type: ignore
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 94, in wrapper_use_tracer
  return func(*args, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/azure/storage/blob/_generated/operations/_service_operations.py", line 849, in get_account_info
  map_error(status_code=response.status_code, response=response, error_map=error_map)
File ".../lib/python3.11/site-packages/azure/core/exceptions.py", line 161, in map_error
  raise error
azure.core.exceptions.ResourceNotFoundError: The specified resource does not exist.
RequestId:...
Time:2024-09-05T08:08:36.4501790Z
ErrorCode:ResourceNotFound
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>ResourceNotFound</Code><Message>The specified resource does not exist.
RequestId:...
Time:2024-09-05T08:08:36.4501790Z</Message></Error>

[pjbull]

Do you have any more info? What permissions does the account you are connecting with have? Are you sure the storage account exists (e.g., is spelled correctly)? Does the same call work if you use the sdk directly?

[M0dEx]

The same test suite passes with cloudpathlib==0.18.1, so as far as the file structure and SDK calls go it should be correct.

Is there any way to check the permissions on a granular basis?
The role of my account is "Storage Blob Data Owner", which means I should have permission to view the account information (and I can see the HNS option in the Azure web GUI just fine).

[M0dEx]

It seems this API endpoint might not support authorization using a provided credential object:

Authorization is required when calling any data access operation in Azure Storage. You can authorize the Get Account Information operation as described below.

This operation doesn't support OAuth-based authorization via an access token from Azure Active Directory/MSI or a user delegation SAS.

https://learn.microsoft.com/en-us/rest/api/storageservices/get-account-information?tabs=shared-access-signatures#authorization

[pjbull]

Ah, ok. It looks like we'll need to find another API call to use as a primary or backup method of determining HNS if we can't get the account info.

[pjbull]

Looks like ResourceNotFound is by design for this API (surprising) if permissions are not sufficient according to Azure/azure-sdk-for-python#21810

We're likely going to need to fallback to checking the blob metadata itself to confirm if there is or isn't HNS.

[pjbull]

@M0dEx Just released v0.20.0 to PyPI. Please, take it for a spin and see if the new approach works for the credentials you're using. Thanks!

[M0dEx]

I can confirm 0.20.0 works just fine with our setup.

Thanks for the update!