Parameters with spaces are split into separate arguments by the arg parser (knack?) even when quoted for bash

[jxn]

This is autogenerated. Please review and update as needed.

Describe the bug

I have a couple of examples of this, but the latest is the using the --team parameter in the az boards iteration teams list command. Instead of taking the quoted arg to interpret it as the property value, if the value contains a space, only the first word is used as the property value, and subsequent words are interpreted as extra args.

Command Name az boards iteration team list --team "MY TEAM NAME WITH SPACES" --timeframe current --debug

debug snippet:

cli.knack.cli: Command arguments: ['boards', 'iteration', 'team', 'list', '--team', 'MY', 'TEAM', 'NAME', 'WITH', 'SPACES', '--timeframe', 'current', '--debug']

Errors:

unrecognized arguments: TEAM NAME WITH SPACES

(the command thinks I passed --team MY, as if I had not properly quoted the parameter value argument)

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Put any pre-requisite steps here...
• az boards iteration team list --team 'MY TEAM WITH A SPACE IN THE NAME' --timeframe current --org=MyOrg --project=MyProject

Expected Behavior

az should interpret the value of --team as "MY TEAM WITH A SPACE IN THE NAME", and pass that encoded value to the api.

Environment Summary

Linux-5.19.0-29-generic-x86_64-with-glibc2.36, Ubuntu 22.10
Python 3.10.7
Installer: PIP

azure-cli 2.44.1

Extensions:
azure-devops 0.26.0

Dependencies:
msal 1.20.0
azure-mgmt-resource 21.1.0b1

Additional Context

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[yonzhan]

route to CXP team

[jsntcy]

@jxn, please check https://learn.microsoft.com/en-us/cli/azure/use-cli-effectively?tabs=bash%2Cbash2#use-quotation-marks-in-parameters to see how to pass string with spaces in bash.

[jxn]

@jxn, please check https://learn.microsoft.com/en-us/cli/azure/use-cli-effectively?tabs=bash%2Cbash2#use-quotation-marks-in-parameters to see how to pass string with spaces in bash.

@jsntcy my example command is included. If you feel there's an error in usage there, could you please be more specific? I looked at the linked article and I don't see anything specific there that is not normal for bash usage. I have been using bash commands, including quoted args, for a few decades, so I do not believe this is due to a failure to me properly quoting, but I'm happy to be proven wrong if you can tell me where my example is wrong.

[SaurabhSharma-MSFT]

@jsntcy Can you please help here.

[jsntcy]

@jiasli, could you please help on this issue?

[jiasli]

I can't reproduce this issue in WSL Ubuntu 22.04:

$ az boards iteration team list --team "MY TEAM NAME WITH SPACES" --timeframe current --debug
cli.knack.cli: Command arguments: ['boards', 'iteration', 'team', 'list', '--team', 'MY TEAM NAME WITH SPACES', '--timeframe', 'current', '--debug']

Where are you using Azure CLI? Could you check which az is being called by running az --version? It should look like

$ az --version
azure-cli                         2.45.0

core                              2.45.0
telemetry                          1.0.8

Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1

Python location '/opt/az/bin/python3'
Extensions directory '/home/user2/.azure/cliextensions'

Also could you verify if this happens when calling other executables?

$ python3 -c "import sys; print(sys.argv)" "MY TEAM NAME WITH SPACES"
['-c', 'MY TEAM NAME WITH SPACES']

[jiasli]

Also can't repro with python:3.10 docker image:

> docker run -it --rm python:3.10 bash

# pip install azure-cli

# az boards iteration team list --team "MY TEAM NAME WITH SPACES" --timeframe current --debug
cli.knack.cli: Command arguments: ['boards', 'iteration', 'team', 'list', '--team', 'MY TEAM NAME WITH SPACES', '--timeframe', 'current', '--debug']

[jxn]

@jiasli here's my version output:

az --version
azure-cli                         2.45.0

core                              2.45.0
telemetry                          1.0.8

Extensions:
azure-devops                      0.26.0

Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1

Python location '/home/jmurtha/.virtualenvs/azure-cli-2.42.0/bin/python3'
Extensions directory '/home/jmurtha/.azure/cliextensions'

Python (Linux) 3.10.7 (main, Nov 24 2022, 19:45:47) [GCC 12.2.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

I think this release has happened since I filed the ticket, so I went ahead and confirmed that the issue is still present, and it is.

I tried to confirm that it's not python itself causing the issue.

first with the system python:

$ python3 -c "import sys; print(sys.argv)" "MY TEAM NAME WITH SPACES"

['-c', 'MY TEAM NAME WITH SPACES']

Then explicitly calling the python used by azure-cli

$ /home/jmurtha/.virtualenvs/azure-cli-2.42.0/bin/python3 -c "import sys; print(sys.argv)" "MY TEAM NAME WITH SPACES"

['-c', 'MY TEAM NAME WITH SPACES']

So, the problem still exists, and it doesn't appear to be python. Perhaps it's linked to some dependency version for a python lib used? Anything I could provide to help investigate that?

[SaurabhSharma-MSFT]

@jiasli can you please look into the information shared by @jxn.

[conjectures]

I think the issue is, in fact, caused by the python argument parser.

I have encountered this issue when I was attempting to pass a string from a variable to the azure cli.

The below works as expected

$ az boards iteration team list --team "MY TEAM" --debug
## debug snippet
## cli.knack.cli: Command arguments: ['boards', 'iteration', 'team', 'list', '--team', 'MY TEAM', '--debug']

If i change the string to a variable, then it doesn't work

$ my_team="MY TEAM"
$ az boards iteration team list --team $my_team --debug
## debug snippet
## cli.knack.cli: Command arguments: ['boards', 'iteration', 'team', 'list', '--team', 'MY', 'TEAM', '--debug']

In fact the above can be reproduced with python:

$ python3 -c "import sys; print(sys.argv)" $my_team
['-c', 'MY', 'TEAM']

My use case was a bit more complicated though. I needed to add multiple fields after the --fields flag and they were surrounded with double quotes as they contained spaces and other symbols. The python parser would pick everything up as a single string and then split it wherever there were spaces. I tried switching the double quotes to single, and then tried escaping double quotes but none of these ideas worked.

My next idea was a bit hacky but it actually worked. I tried to transform the string into a bash array and then expanding it. I am not sure why it works yet but it seems that the python interpreter doesn't confuse it as a string in this case.

$ my_team="MY TEAM"

## Create array from string
$ readarray -t team_array < <(echo $my_team)  

## Expand array 
$ python3 -c "import sys; print(sys.argv)" "${team_array[0]}"
['-c', 'MY TEAM']

In my use case I would add all the 'field strings' in array elements and then expand fully in the az command:

## create 'complex' string of fields (notice single quote in the beginning)
$ fields='"Category=SE"\n"Urgency=10"'

## add fields to array. ( sed used to remove double quotes)
$ readarray -t fields_array < <(echo -e $fields | sed 's/"//g')
 
 ## we can check if the fields are stored correctly:
$ declare -p fields_array
declare -a fields_array=([0]="Category=SE" [1]="Urgency=10")

## Create work item with fields in azure
$ az boards work-item create --project "my-project" --title "Test" --type "Bug" --fields "${fields_array[@]}" --debug

## cli.knack.cli: Command arguments: ['boards', 'work-item', 'create', '--project', 'my-project', '--title', 'Test', '--type', 'Bug', '--fields', 'Category=SE', 'Urgency=10', '--debug']