Image copy, subprocess.CalledProcessError returned non-zero exit status 3.

[DevOpsDAdams]

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name az image copy Extension Name: image-copy-extension. Version: 0.2.7.

Errors:

Command '['/usr/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Windows-2019-Crowe-Template-NCUS-PRD1_os_disk_snapshot', '--location', 'northcentralus', '--resource-group', 'IS-CoreSystem-PRD1', '--source', '/subscriptions/<<removed>>/resourceGroups/TEMP-image-transfer/providers/Microsoft.Compute/snapshots/Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus', '--output', 'json', '--tags', 'created_by=image-copy-extension']' returned non-zero exit status 3.
Traceback (most recent call last):
...

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Put any pre-requisite steps here...
• az image copy --source-object-name {} --source-resource-group {} --target-location {} --target-resource-group {} --target-subscription {} --target-name {}

Expected Behavior

Environment Summary

Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
Python 3.6.8
Installer: RPM

azure-cli 2.13.0

Extensions:
log-analytics 0.2.1
image-copy-extension 0.2.7

Additional Context

[yungezz]

hi @qwordy could you pls help to have a look? thanks

[qwordy]

@dadams0641 Can you show the traceback? Thanks.

[DevOpsDAdams]

Sure!

Getting OS disk ID of the source VM/image
Creating source snapshot
command failed: ['/usr/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Windows-2019-Crowe-Template-NCUS-PRD1_os_disk_snapshot', '--location', 'northcentralus', '--resource-group', 'IS-CoreSystem-PRD1', '--source', '/subscriptions/<<removed>>/resourceGroups/TEMP-image-transfer/providers/Microsoft.Compute/snapshots/Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus', '--output', 'json', '--tags', 'created_by=image-copy-extension']
output: ResourceNotFoundError: Resource Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus is not found.

CLIInternalError: The command failed with an unexpected error. Here is the traceback:
Command '['/usr/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Windows-2019-Crowe-Template-NCUS-PRD1_os_disk_snapshot', '--location', 'northcentralus', '--resource-group', 'IS-CoreSystem-PRD1', '--source', '/subscriptions/<<removed>>/resourceGroups/TEMP-image-transfer/providers/Microsoft.Compute/snapshots/Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus', '--output', 'json', '--tags', 'created_by=image-copy-extension']' returned non-zero exit status 3.
Traceback (most recent call last):
  File "/usr/lib64/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute
    raise ex
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 711, in _run_job
    six.reraise(*sys.exc_info())
  File "/usr/lib64/az/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/lib64/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 784, in default_command_handler
    return op(**command_args)
  File "/home/resadmin/.azure/cliextensions/image-copy-extension/azext_imagecopy/custom.py", line 105, in imagecopy
    run_cli_command(cli_cmd)
  File "/home/resadmin/.azure/cliextensions/image-copy-extension/azext_imagecopy/cli_utils.py", line 45, in run_cli_command
    raise ex
  File "/home/resadmin/.azure/cliextensions/image-copy-extension/azext_imagecopy/cli_utils.py", line 21, in run_cli_command
    cmd_output = check_output(cmd, stderr=STDOUT, universal_newlines=True)
  File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Windows-2019-Crowe-Template-NCUS-PRD1_os_disk_snapshot', '--location', 'northcentralus', '--resource-group', 'IS-CoreSystem-PRD1', '--source', '/subscriptions/<<removed>>/resourceGroups/TEMP-image-transfer/providers/Microsoft.Compute/snapshots/Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus', '--output', 'json', '--tags', 'created_by=image-copy-extension']' returned non-zero exit status 3.

[qwordy]

It says "Resource Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus is not found". Can you check if it exists?

[DevOpsDAdams]

So, that's what I was thinking at first, maybe I got the name, subscription, or RG incorrect, but so far as I can tell, I got them right.

Here's the command I ran:

az image copy --source-object-name "Windows-2019-Crowe-Template-NCUS-PRD1" --source-resource-group "IS-CoreSystem-PRD1" --target-location "North Central US" --target-resource-group "IS-CoreSystem-QA1" --target-subscription "Crowe QA 1" --target-name "Windows-2019-Crowe-Template-NCUS-QA1" --verbose

Here's a screenshot of the source image. It exists and I double checked my spelling, just in case:

With ResourceID:

/subscriptions/<<removed>>/resourceGroups/IS-CoreSystem-PRD1/providers/Microsoft.Compute/images/Windows-2019-Crowe-Template-NCUS-PRD1

EDIT: I am trying now by setting the resourceID as the source-object-name (even though that's not what the documentation calls for) and I get a different error that directs me to this "https://aka.ms/ARMResourceNotFoundFix"

However none of those solutions seem to apply to this. I've got a little bit of down time today, I'll give it another go with some alternatives and see what I dig up

[michaelschmit]

I am running into this same issue. What is likely happening is that the original OS disk that was used to create the image no longer exists. This error is occurring for me as well because I am using packer to generate the image. Packer creates a temporary resource group, creates a VM, takes an image, then puts the image in the specified resource group. Finally, Packer cleans up the temporary VM and resource group. When the copy image command runs it tries to locate the original OS disk that no longer exists and fails. Perhaps there would be a way to output a better error message to help people who run into this issue as a short term fix. For a long term fix it would be nice to not have to require the original OS disk, but I realize there are likely some azure constraints to work around. It would be great if azure had the ability to just clone an image rather than these other games that are being played.

[qwordy]

A similar issue https://github.com/Azure/azure-cli-extensions/issues/1756. I am still trying to find the cause. What if you run the failed command manually?

subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Windows-2019-Crowe-Template-NCUS-PRD1_os_disk_snapshot', '--location', 'northcentralus', '--resource-group', 'IS-CoreSystem-PRD1', '--source', '/subscriptions/<<removed>>/resourceGroups/TEMP-image-transfer/providers/Microsoft.Compute/snapshots/Windows-2019-Crowe-Template_os_disk_snapshot-northcentralus', '--output', 'json', '--tags', 'created_by=image-copy-extension']' returned non-zero exit status 3.

Run command like

az snapshot create --name Windows-2019-Crowe-Template-NCUS-PRD1_os_disk_snapshot ...

[qwordy]

I am running into this same issue. What is likely happening is that the original OS disk that was used to create the image no longer exists. This error is occurring for me as well because I am using packer to generate the image. Packer creates a temporary resource group, creates a VM, takes an image, then puts the image in the specified resource group. Finally, Packer cleans up the temporary VM and resource group. When the copy image command runs it tries to locate the original OS disk that no longer exists and fails. Perhaps there would be a way to output a better error message to help people who run into this issue as a short term fix. For a long term fix it would be nice to not have to require the original OS disk, but I realize there are likely some azure constraints to work around. It would be great if azure had the ability to just clone an image rather than these other games that are being played.

Totally agree. This extension is full of magic. It is a very long path to copy a image. I suggest Azure service support image copy operation. But unfortunately, they are all devoted to shared image gallery. It is an advanced version of image management. You can have a try. Need some learning time.

[qwordy]

I can't reproduce it and find cause so far.

[yungezz]

image-copy extension need to be improved systematically, add feature-request to this one.

[ghost]

This bug was never fixed and it appears that the az image copy command isn't able to function at all - over a year later from these comments. It isn't a feature request; I don't think that can be disputed. It is a feature that exists and has been broken for years now without being fixed.

Before anyone says that you need the disk to use this command, that is NOT what the documentation says. The source can be a disk OR an image. Lots and lots of people making incorrect assumptions about this command or the documentation is just plain wrong.

I'm just going to leave this hear as someone is going to say it anyway. This command supports both disk and images and It will default to the type of image, not disk - according to the documentation:

--source-type Image or vm.

accepted values: image, vm default value: image

Correct command as specified by the documentation:

      az image copy --source-resource-group CustomImagesRG `
      --source-object-name Win10MultiV9.5-20220107 `
      --target-resource-group CustomImagesRG `
      --target-location eastus `
      --target-subscription 12345678-1234-1234-1234-605185bb3005 

Output:

Getting OS disk ID of the source VM/image Creating source snapshot command failed: ['/usr/bin/../../opt/az/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Win10MultiV9.5-20220107_os_disk_snapshot', '--location', 'eastus', '--resource-group', 'CustomImagesRG', '--source', '/subscriptions/12345678-1234-1234-1234-605185bb3005/resourceGroups/AVDImageBuildRG/providers/Microsoft.Compute/disks/AVDImageBuild_OsDisk_1_4125c892f47a42c09530358ab9632b88', '--hyper-v-generation', 'V1', '--output', 'json', '--tags', 'created_by=image-copy-extension'] output: ERROR: (NotFound) Resource AVDImageBuild_OsDisk_1_4125c892f47a42c09530358ab9632b88 is not found. Code: NotFound Message: Resource AVDImageBuild_OsDisk_1_4125c892f47a42c09530358ab9632b88 is not found.

The command failed with an unexpected error. Here is the traceback: Command '['/usr/bin/../../opt/az/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Win10MultiV9.5-20220107_os_disk_snapshot', '--location', 'eastus', '--resource-group', 'CustomImagesRG', '--source', '/subscriptions/12345678-1234-1234-1234-605185bb3005/resourceGroups/AVDImageBuildRG/providers/Microsoft.Compute/disks/AVDImageBuild_OsDisk_1_4125c892f47a42c09530358ab9632b88', '--hyper-v-generation', 'V1', '--output', 'json', '--tags', 'created_by=image-copy-extension']' returned >> non-zero exit status 3. Traceback (most recent call last): File "/opt/az/lib/python3.10/site-packages/knack/cli.py", line 231, in invoke cmd_result = self.invocation.execute(args) File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 663, in execute raise ex File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 697, in _run_job result = cmd_copy(params) File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/init.py", line 333, in call return self.handler(*args, **kwargs) File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler return op(**command_args) File "/home/csp/.azure/cliextensions/image-copy-extension/azext_imagecopy/custom.py", line 108, in imagecopy run_cli_command(cli_cmd) File "/home/csp/.azure/cliextensions/image-copy-extension/azext_imagecopy/cli_utils.py", line 45, in run_cli_command raise ex File "/home/csp/.azure/cliextensions/image-copy-extension/azext_imagecopy/cli_utils.py", line 21, in run_cli_command cmd_output = check_output(cmd, stderr=STDOUT, universal_newlines=True) File "/opt/az/lib/python3.10/subprocess.py", line 420, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/opt/az/lib/python3.10/subprocess.py", line 524, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/../../opt/az/bin/python3', '-m', 'azure.cli', 'snapshot', 'create', '--name', 'Win10MultiV9.5-20220107_os_disk_snapshot', '--location', 'eastus', '--resource-group', 'CustomImagesRG', '--source', '/subscriptions/12345678-1234-1234-1234-605185bb3005/resourceGroups/AVDImageBuildRG/providers/Microsoft.Compute/disks/AVDImageBuild_OsDisk_1_4125c892f47a42c09530358ab9632b88', '--hyper-v-generation', 'V1', '--output', 'json', '--tags', 'created_by=image-copy-extension']' returned non-zero exit status 3.