aws-cli
aws-cli copied to clipboard
aws lambda list-functions generate-cli-skeleton differs from its help output
Describe the bug
The pagination options emitted by --generate-cli-skeleton
for list-functions
are different from its actual invocation
aws lambda list-functions --generate-cli-skeleton
{
"MasterRegion": "",
"FunctionVersion": "ALL",
"Marker": "",
"MaxItems": 0
}
$ AWS_PAGER=cat aws lambda list-functions help | sed -ne "/^.*SYNOPSIS/,/^.*OPTIONS/p"
SYNOPSIS
list-functions
[--master-region <value>]
[--function-version <value>]
[--max-items <value>]
[--cli-input-json | --cli-input-yaml]
[--starting-token <value>]
[--page-size <value>]
[--generate-cli-skeleton <value>]
Observe the --starting-token
NOT --marker
Expected Behavior
That the --generate-cli-skeleton
would be the programmatic representation of the input arguments
Current Behavior
see above
Reproduction Steps
see above (or below)
Possible Solution
No response
Additional Information/Context
I can reproduce this with the docker image, so it is not just on my machine:
$ docker run --rm amazon/aws-cli:2.5.4 lambda list-functions --generate-cli-skeleton
{
"MasterRegion": "",
"FunctionVersion": "ALL",
"Marker": "",
"MaxItems": 0
}
$ docker run --rm amazon/aws-cli:2.5.4 lambda list-functions help | sed -ne '/^.*SYNOP/,/^.*OPTION/p'
SYNOPSIS
list-functions
[--master-region <value>]
[--function-version <value>]
[--max-items <value>]
[--cli-input-json | --cli-input-yaml]
[--starting-token <value>]
[--page-size <value>]
[--generate-cli-skeleton <value>]
CLI version used
aws-cli/2.5.4 Python/3.9.12 Linux/5.13.0-39-generic source/x86_64.ubuntu.21 prompt/off
Environment details (OS name and version, etc.)
Ubuntu 21.10, "brew install -s awscli"
Hi @mdaniel thanks for reaching out. The model for this paginator is defined here. There are some inconsistencies in how service APIs name their markers/tokens. Marker is a request parameter in the underlying API but it is abstracted to NextMarker
for pagination.
The --starting-token
is described in the documentation for this command here:
A token to specify where to start paginating. This is the
NextToken
from a previously truncated response.
So the --starting-token
is only needed if there was a NextToken
from the previous response. For general information on pagination using the CLI you can refer to this page in the User Guide.
I also created a tracking issue here for improvements to the functionality/documentation of --generate-cli-skeleton
. We can include the pagination behavior as something that could be better documented as it relates to --generate-cli-skeleton
. I hope that helps, please let us know if you have any questions or feedback.
Thank you for linking to the underlying issue
However, through what mechanism can a user learn what value to put into {"Marker":""}
since the list-function
output contains {"NextToken":""}
? edit: well, I guess given the experiment below, the answer is to only use one style or the other, --style
or --cli-input-json && --generate-cli-skeleton
and not try and mix the two :-/
I thought perhaps you meant --generate-cli-skeleton
was merely misconfigured but I had bad outcomes from some "just try it"s:
$ aws lambda list-functions --max-items 2
{
"Functions": [
{
"FunctionName": ...
}
],
"NextToken": "eyJNYXJrZXIiOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAyfQ=="
}
$ aws lambda list-functions --cli-input-json '{"StartingToken": "eyJNYXJrZXIiOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAyfQ=="}'
Parameter validation failed:
Unknown parameter in input: "StartingToken", must be one of: MasterRegion, FunctionVersion, Marker, MaxItems
$ aws lambda list-functions --cli-input-json '{"Marker": "eyJNYXJrZXIiOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAyfQ=="}'
An error occurred (InvalidParameterValueException) when calling the ListFunctions operation: Invalid marker value: eyJNYXJrZXIiOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAyfQ==
While testing this, I found even more weird behavior; if one uses --max-items
one gets {"NextToken":""}
in the output, but if one uses {"MaxItems":2}
the response contains the {"NextMarker":""}
as documented by the OUTPUT
section of the help
$ aws lambda list-functions --cli-input-json '{"MaxItems": 2}'
{
"NextMarker": "bAsE64==",
"Functions": [
{}
]
}
I think this comment on an older issue might help clear up some of the behavior around pagination: https://github.com/aws/aws-cli/issues/2555#issuecomment-295825435
For more information on --generate-cli-skeleton
and --cli-input-json
I would refer to this page in the User Guide: https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-skeleton.html
I couldn’t reproduce the behavior you described from running this command: aws lambda list-functions --cli-input-json '{"MaxItems": 2}'
. If you provide the debug logs by adding --debug
to the command then we could look into it further.
based on --debug
, the major difference is that --max-items
filters client side whereas {"MaxItems":2}
filters them server side
$ aws --debug --region ca-central-1 lambda list-functions --cli-input-json '{"MaxItems":2}' 2>github-6870.err.log | jq '.|keys'
[
"Functions",
"NextMarker"
]
$ aws --debug --region ca-central-1 lambda list-functions --max-items 2 2> github-6870.dash-args.log | jq '.|keys'
[
"Functions",
"NextToken"
]
Checking back in here. As mentioned previously there are inconsistencies between how service APIs define their pagination tokens. The Lambda ListFunctions API uses NextMarker
as you demonstrated but the CLI pagination uses NextToken
. And your example does show the server-side vs client-side distinction in behavior.
You could disable the CLI pager as described in the documentation if that would help with your use case. If you have any feedback on how the CLI pagination documentation could be improved please let us know here. And for improvements regarding generate-cli-skeleton
we can continue tracking those in https://github.com/aws/aws-cli/issues/6695 if you want to add any comments to that issue.
Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.