azure-cli
azure-cli copied to clipboard
az storage fs directory upload with login authentication fails with MSI aut not yet supported
This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az storage fs directory upload
Errors:
The command failed with an unexpected error. Here is the traceback:
MSI auth not yet supported.
Traceback (most recent call last):
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 108, in login_auth_for_azcopy
token_info = _unserialize_non_msi_token_payload(token_info)
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 144, in _unserialize_non_msi_token_payload
parsed_authority = urlparse(token_info['_authority'])
KeyError: '_authority'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 231, in invoke
cmd_result = self.invocation.execute(args)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 657, in execute
raise ex
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 720, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 712, in _run_job
return cmd_copy.exception_handler(ex)
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 371, in new_handler
first(ex)
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/__init__.py", line 370, in new_handler
raise ex
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 691, in _run_job
result = cmd_copy(params)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
return self.handler(*args, **kwargs)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
return op(**command_args)
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/azcopy.py", line 74, in storage_fs_directory_copy
azcopy = _azcopy_login_client(cmd)
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/operations/azcopy.py", line 125, in _azcopy_login_client
return AzCopy(creds=login_auth_for_azcopy(cmd))
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 110, in login_auth_for_azcopy
raise Exception('MSI auth not yet supported.')
Exception: MSI auth not yet supported.
To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.
- authentication with
az login --use-device-code
using an AD account -
az storage fs directory upload --file-system {} --account-name {} --auth-mode {} --destination-path {} --recursive --source {}
Expected Behavior
Directory to be uploaded successfully. It worked with version 2.27.2 of az cli on August 20th.
Environment Summary
Linux-4.9.0-13-amd64-x86_64-with-debian-9.13, Debian GNU/Linux 9 (stretch)
Python 3.6.10
Installer: DEB
azure-cli 2.30.0
Extensions:
storage-preview 0.7.4
Additional Context
target is an ADLS Gen 2 Data Lake
storage
Meet the same issue when calling "az storage blob directory upload" azure cli version : 2.30.0 storage-preview version : 0.7.1
Error: WARNING: This command is implicitly deprecated because command group 'storage blob directory' is deprecated and will be removed in a future release. Use 'az storage fs directory' instead. ERROR: The command failed with an unexpected error. Here is the traceback: ERROR: MSI auth not yet supported. Traceback (most recent call last): File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview\azcopy\util.py", line 81, in blob_client_auth_for_azcopy token_info = _unserialize_non_msi_token_payload(token_info) File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview\azcopy\util.py", line 104, in _unserialize_non_msi_token_payload parsed_authority = urlparse(token_info['_authority']) KeyError: '_authority'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 231, in invoke File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 657, in execute File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 720, in run_jobs_serially File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 712, in run_job File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview_init.py", line 252, in new_handler handler(ex) File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview_init.py", line 194, in handler raise ex File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 691, in _run_job File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 328, in call File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview\operations\azcopy.py", line 19, in storage_blob_upload azcopy = _azcopy_blob_client(cmd, client) File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview\operations\azcopy.py", line 65, in _azcopy_blob_client return AzCopy(creds=blob_client_auth_for_azcopy(cmd, client)) File "C:\Users\shiswang.azure\cliextensions\storage-preview\azext_storage_preview\azcopy\util.py", line 83, in blob_client_auth_for_azcopy raise Exception('MSI auth not yet supported.') Exception: MSI auth not yet supported. To open an issue, please run: 'az feedback'
az storage fs directory upload
is using azcopy
which is integrated inside Azure CLI. After CLI core migrated to MSAL, MSAL won't return refresh_token
any more. This causes authentication failure with azcopy when users specify --auth-mode login
.
Please try these workarounds:
-
az storage fs directory --file-system {} --account-name {} --destination-path {} --recursive --source {}
-
az storage fs directory --file-system {} --account-name {} --destination-path {} --recursive --source {} --auth-mode key –account-key {}
-
az storage fs directory --file-system {} --account-name {} --destination-path {} --recursive --source {} --connection-string {}
-
az storage fs directory --file-system {} --account-name {} --destination-path {} --recursive --source {} --sas-token {}
We are working with MSAL team to see if they can add refresh_token
in response and seek advice from Azcopy team if they can use MSAL cache directly.
Thank you for the suggestions, however authenticating with an AAD account is what I need, as in my context I currently don't have the other authentication options. ACLs are setup on my AAD account and I can't generate a SAS token, nor do have I access to an account key for the target storage.
In my case where I needed to transfer files from a remote server to the storage, I ended up having to download them to my desktop and upload them to the storage through Azure Storage Explorer :(
Sorry for the inconvenience. If you have to auth with AAD, can you try azcopy directly instead of using cli's integrated azcopy?
Sorry for the inconvenience. If you have to auth with AAD, can you try azcopy directly instead of using cli's integrated azcopy?
Ran a quick test and it did work with direct azcopy
, thanks for the suggestion :)
For me it worked to provide the AZCOPY vars for SP client secret login, since the fs directory uses az copy internally this resolved the auth issue for me.
PS: I am using 'az storage fs directory download' exporting these before hand works:
export AZCOPY_AUTO_LOGIN_TYPE=SPN export AZCOPY_SPA_APPLICATION_ID=<some_sp_app_id> export AZCOPY_SPA_CLIENT_SECRET=<some_password> export AZCOPY_TENANT_ID=<some_tenand_id>
Also getting this issue trying to run az storage azcopy
after using OIDC login (https://github.com/Azure/communication-ui-library/actions/runs/9034838453/job/24828300265#step:19:70)