az image copy fails with 'command ended with an error'

[RickKra]

Command Name
az image copy Extension Name: image-copy-extension. Version: 0.2.1.

Errors:

Starting async process for all locations
westus2 - Creating target storage account (can be slow sometimes)
command ended with an error: ['C:\\Program Files (x86)\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'azure.cli', 'storage', 'account', 'create', '--name', 'westus2c7eafby7ltzpp1393', '--resource-group', 'image-copy-Nxs2019100102', '--location', 'westus2', '--sku', 'Standard_LRS', '--output', 'json', '--subscription', 'vsclk-core-dev', '--tags', 'created_by=image-copy-extension']
Deleting transient resources

I tried it with --debug and that didn't give any additional info about the error.

To Reproduce:

az image copy --target-subscription vsclk-core-dev --target-resource-group vsclk-online-dev --target-name Nxs2019100102 --target-location westus2 --source-resource-group VSNexusImages --source-object-name Nxs2019100102 --cleanup --timeout 36000 --temporary_resource_group_name image-copy-Nxs2019100102

Expected Behavior

Copies the image to the target sub/rg.

Environment Summary

Windows-10-10.0.18362-SP0
Python 3.6.6
Shell: cmd.exe

azure-cli 2.0.74

Extensions:
azure-devops 0.12.0
image-copy-extension 0.2.1

Additional Context

I was getting this failure in an Azure DevOps release pipeline using the Azure CLI task.

I thought at first it might be related to using an Azure service principal or an Azure pipeline to do this, because I've been able to successfully do this myself locally.

However, one of the last mitigations I attempted was to update the Azure CLI locally and try that. I went from 2.0.70 to 2.0.74, and now I'm getting the same error locally.

The Azure CLI task was using 2.0.46 and also failing, so I tried updating the CLI using the automation suggested at https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest#install-or-update. That updated the CLI on the build agent to 2.0.74 and it also failed there again.

Hoping you've got a way to install the 2.0.70 CLI via automation so I can work around this until you fix it?

Thanks,
Rick

[yonzhan]

@qwordy please take a look and comment.

[qwordy]

Received. National day holiday: 1st Oct ~ 7th Oct.

[jmhardison]

Ran into this, stumbled on this issue with a colleague.

This is failing due to the az cli command now outputting:
"WARNING: The default kind for created storage account will change to 'StorageV2' from 'Storage' in future"

This is noted over here: Azure/azure-cli#10687

As a very ugly workaround, the cli_utils.py can be modified to clean it up.

if return_as_json:
            if cmd_output:
                # temp cleanup to deal with new warning string  - issue #979
                # Related to: https://github.com/Azure/azure-cli/issues/10687
                warn_string = "WARNING: The default kind for created storage account will change to 'StorageV2' from 'Storage' in future"
                cmd_output = cmd_output.replace(warn_string, "")
                json_output = json.loads(cmd_output)
                return json_output

I'll submit a pr to help with this, but root cause is not this extension. And even changing to use StorageV2 doesn't remove the warning, which should not be output as a random string when you desire JSON. But again, not the fault of this extension.

Otherwise the short workaround code above can be applied by a user to their current installed extension.

[tamirkamara]

I believe this issue is resolved with version 0.2.2 of the extension.
@zikalino please close it.

[ggorman]

I'm still having this problem. Running on the azure portal bash shell. Just installed image-copy-externsion.

  "extensionType": "whl",
  "metadata": {
    "azext.minCliCoreVersion": "2.0.24",
    "classifiers": [
      "Development Status :: 4 - Beta",
      "Intended Audience :: Developers",
      "Intended Audience :: System Administrators",
      "Programming Language :: Python",
      "Programming Language :: Python :: 2",
      "Programming Language :: Python :: 2.7",
      "Programming Language :: Python :: 3",
      "Programming Language :: Python :: 3.4",
      "Programming Language :: Python :: 3.5",
      "Programming Language :: Python :: 3.6",
      "License :: OSI Approved :: MIT License"
    ],
    "extensions": {
      "python.details": {
        "contacts": [
          {
            "email": "[email protected]",
            "name": "Tamir Kamara",
            "role": "author"
          }
        ],
        "document_names": {
          "description": "DESCRIPTION.rst"
        },
        "project_urls": {
          "Home": "https://github.com/Azure/azure-cli-extensions"
        }
      }
    },
    "generator": "bdist_wheel (0.30.0)",
    "license": "MIT",
    "metadata_version": "2.0",
    "name": "image-copy-extension",
    "summary": "Support for copying managed vm images between regions",
    "version": "0.2.1"
  },
  "name": "image-copy-extension",
  "version": "0.2.1"
}

I cannot find any documentation on how specify a specific version of an extension.

[ggorman]

@tamirkamara @zikalino - it seems premature to close this. I notice that image-copy-extension 0.2.2 has not yet been merged due to failing travis errors. See PR #1108

[ggorman]

Upgraded to prelease of extension using
pip install --upgrade --target ~/.azure/cliextensions/image-copy-extension "git+https://github.com/Azure/azure-cli-extensions.git#subdirectory=src/image-copy&egg=image-copy-extension"
Now getting the error:
output: ERROR: Required parameter 'storageAccountId' is missing (null).

[arrownj]

@ggorman I think the original copy image issue is already fixed in 0.2.2. And the new image-copy-extension version has already been released. You can use az extension update to upgrade it.

[jonathanclarke]

This is still an ongoing issue:
azure-cli 2.0.78

Extensions:
image-copy-extension 0.2.2
output: ERROR: Required parameter 'storageAccountId' is missing (null).

[tamirkamara]

@jonathanclarke your issue isn't like the original one opened in this issue.
please see #1097 (hint, use cli version 76)

@zikalino can we close this issue please?

[tamirkamara]

@qwordy can you close this issue?