aws-cli
aws-cli copied to clipboard
`aws batch list-jobs`: inconsistent JSON object returned; `NextToken`
Describe the bug
There is a naming inconsistency in the JSON
objects returned for aws batch list-jobs
, specifically the nextToken
is named with lower case and then in capital case in some instances with different flag combinations passed.
Expected Behavior
I expect for AWS Batch list-jobs
to return the same structured (same names on keys) every time no matter the combination of flags passed.
Either NextToken
or nextToken
, but it needs to be consistent.
Current Behavior
When I use this command I get this back:
aws batch list-jobs --job-queue $QUEUE --job-status $JOB_STATUS --no-paginate
...
{
"jobArn": "arn:aws:batch:us-east-1::job/791ce716-4563",
"jobId": "3cc4-4563-8b0a",
"jobName": "dereck-fasterq-dump-20220830-202208",
"createdAt": 1661890929465,
"status": "RUNNABLE"
}
],
"nextToken": "Ka9qeIy8WcVDfTPlUp/wxF1NCNUSr5ZSczSoaXNzxe9IQ5MC3RnhYlIfmpofNyZZ/hJDnDd8jUGtGV1uILxTQzYCyT6ONeaL9XK1Kv9eS9POM4d1qCw9cZSdASocllknghuVP9uU7Mr/L+cAsfKallUT+J97UfmM/cyxtLrnsBLd8yQqAUzXQEe+dguaSDIYFUHNNhk1DNIoCj2lymSWMwWGGHg4MZu1H3qdhgKh+fTgKcIKi2drxeHinMm3KEfoVrwFuG8Efk7K/NPOjZA55DqSuGPBjRkFoebOTX/sLbYgV9s9sQDLxTOg4NO6tvFhfFKrKBVHtFAnp0Y6z1rP9ZyNTEKvwKuxNAkhMrH5ttiWCSRlH5CWqRxCe9rD0eMNWcnHgt7Rx/sgDX2O8NJaypJofwmkP0t4irVNnbqcuwxYXasTqhSwFeKrVplu4ndYBApKg0Npltqd0RvEKBhOA4+rsLwy1EoHubeHA0hS9J4lpXfld1TBwBk3/+aicPDL1/0xNQrlQ+AiMtZV/ppyHuG9k7yXknNGH25lV1O9Q0PBOj24igYagHDeGyTGdhIadUbMsvIpP4Y0t8P8zzjuaeoWL9ccwDfyALdgYsJQ17KtzKr7XFuw/5nBDbOas/z+XfIcNhUI3bIC3AlxQAMziWdduGXKo1E9yUL7k7M6Z1f11F9uTDi5DyaIOxuloWFWoECgGnchLeVqovbKg8KYXbtuOlYOCEWl0o/f9sViMl2jXanTsB3IjJWovipl1XCcIDCn1O+Z3FaYcqn4FfZ//884ZXo2iNzzWQoJ331Bk/TgKcIKi2drxeHinMm3KEfoVrwFuG8Efk7K/NPOjZA55DqSuGPBjRkFoebOTX/sLbYgV9s9sQDLxTOg4NO6tvFhfFKrKBVHtFAnp0Y6z1rP9ZyNTEKvwKuxNAkhMrH5ttgSxxq0NM4Jp1P5yj3auhwPJZ3mIiRq8sTrSBBh3yk/ww6zmOAs+szybBznPtDbA7nqOeDgbJ9r8jk+y/rEwk/kDeiVXxtG+zeNlEk+paKuKA3olV8bRvs3jZRJPqWirigN6JVfG0b7N42UST6loq4oDeiVXxtG+zeNlEk+paKuKMgOALdOc9LEeK2HFgjC0c0="
}
But when I do this I get this result back:
aws batch list-jobs --job-queue $QUEUE --job-status $JOB_STATUS --max-items 10
...
},
{
"jobArn": "arn:aws:batch:us-east-1::job/374a1d2f-5d01-b034",
"jobId": "374a1d2f-46f3-b034",
"jobName": "dereck-fasterq-dump-20220830-195637",
"createdAt": 1661889398756,
"status": "RUNNABLE"
}
],
"NextToken": "eyJuZXh0VG9rZW4iOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAxMH0="
}
My goal is to chain with jq
to extract information and the next page token:
aws batch list-jobs --job-queue $QUEUE --job-status $JOB_STATUS --max-items 1000 | jq -r '.NextToken'
The key is named inconsistently which prevents me from doing that when using the flag --no-paginate
. When the --no-paginate
flag is passed the key is nextToken
and when the --max-items
flag (or anything else) is passed then the key is named NextToken
.
Reproduction Steps
Set up a job queue and submit some batch jobs. Then run the above commands.
Possible Solution
Capitalise all NextToken
for consistency.
Additional Information/Context
No response
CLI version used
aws-cli/2.7.25 Python/3.9.11 Linux/5.4.0-1081-aws exe/x86_64.ubuntu.18 prompt/off
Environment details (OS name and version, etc.)
Ubuntu Server 18.04 LTS (HVM)
Hi @dereckdemezquita thanks for reaching out. The ListJobs API returns nextToken
, but when using --max-items
with the CLI pagination it will return NextToken
. For more explanation on the --no-paginate
behavior you can refer to this comment:
So in general for AWS API's, they will return a next token to indicate if there are more results to return. If you call
--no-paginate
, it will return the response from a single API request, which will have a next token if not all of the resources are returned in that single response. The CLI by default (and possible) will make the subsequent API calls for you and since it continues paginating until there are no more resource left to list, it will not include a next token in the response displayed because it listed through all of the resources.
Since we haven't heard back in a few weeks I'm going to close this issue. If you are still affected by this please refer to the comment above and follow up here. Thanks!
⚠️COMMENT VISIBILITY WARNING⚠️
Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.