`aws batch list-jobs`: inconsistent JSON object returned; `NextToken`

[dereckmezquita]

Describe the bug

There is a naming inconsistency in the JSON objects returned for aws batch list-jobs, specifically the nextToken is named with lower case and then in capital case in some instances with different flag combinations passed.

Expected Behavior

I expect for AWS Batch list-jobs to return the same structured (same names on keys) every time no matter the combination of flags passed.

Either NextToken or nextToken, but it needs to be consistent.

Current Behavior

When I use this command I get this back:

aws batch list-jobs --job-queue $QUEUE --job-status $JOB_STATUS --no-paginate

...
        {
            "jobArn": "arn:aws:batch:us-east-1::job/791ce716-4563",
            "jobId": "3cc4-4563-8b0a",
            "jobName": "dereck-fasterq-dump-20220830-202208",
            "createdAt": 1661890929465,
            "status": "RUNNABLE"
        }
    ],
    "nextToken": "Ka9qeIy8WcVDfTPlUp/wxF1NCNUSr5ZSczSoaXNzxe9IQ5MC3RnhYlIfmpofNyZZ/hJDnDd8jUGtGV1uILxTQzYCyT6ONeaL9XK1Kv9eS9POM4d1qCw9cZSdASocllknghuVP9uU7Mr/L+cAsfKallUT+J97UfmM/cyxtLrnsBLd8yQqAUzXQEe+dguaSDIYFUHNNhk1DNIoCj2lymSWMwWGGHg4MZu1H3qdhgKh+fTgKcIKi2drxeHinMm3KEfoVrwFuG8Efk7K/NPOjZA55DqSuGPBjRkFoebOTX/sLbYgV9s9sQDLxTOg4NO6tvFhfFKrKBVHtFAnp0Y6z1rP9ZyNTEKvwKuxNAkhMrH5ttiWCSRlH5CWqRxCe9rD0eMNWcnHgt7Rx/sgDX2O8NJaypJofwmkP0t4irVNnbqcuwxYXasTqhSwFeKrVplu4ndYBApKg0Npltqd0RvEKBhOA4+rsLwy1EoHubeHA0hS9J4lpXfld1TBwBk3/+aicPDL1/0xNQrlQ+AiMtZV/ppyHuG9k7yXknNGH25lV1O9Q0PBOj24igYagHDeGyTGdhIadUbMsvIpP4Y0t8P8zzjuaeoWL9ccwDfyALdgYsJQ17KtzKr7XFuw/5nBDbOas/z+XfIcNhUI3bIC3AlxQAMziWdduGXKo1E9yUL7k7M6Z1f11F9uTDi5DyaIOxuloWFWoECgGnchLeVqovbKg8KYXbtuOlYOCEWl0o/f9sViMl2jXanTsB3IjJWovipl1XCcIDCn1O+Z3FaYcqn4FfZ//884ZXo2iNzzWQoJ331Bk/TgKcIKi2drxeHinMm3KEfoVrwFuG8Efk7K/NPOjZA55DqSuGPBjRkFoebOTX/sLbYgV9s9sQDLxTOg4NO6tvFhfFKrKBVHtFAnp0Y6z1rP9ZyNTEKvwKuxNAkhMrH5ttgSxxq0NM4Jp1P5yj3auhwPJZ3mIiRq8sTrSBBh3yk/ww6zmOAs+szybBznPtDbA7nqOeDgbJ9r8jk+y/rEwk/kDeiVXxtG+zeNlEk+paKuKA3olV8bRvs3jZRJPqWirigN6JVfG0b7N42UST6loq4oDeiVXxtG+zeNlEk+paKuKMgOALdOc9LEeK2HFgjC0c0="
}

But when I do this I get this result back:

aws batch list-jobs --job-queue $QUEUE --job-status $JOB_STATUS --max-items 10

...
        },
        {
            "jobArn": "arn:aws:batch:us-east-1::job/374a1d2f-5d01-b034",
            "jobId": "374a1d2f-46f3-b034",
            "jobName": "dereck-fasterq-dump-20220830-195637",
            "createdAt": 1661889398756,
            "status": "RUNNABLE"
        }
    ],
    "NextToken": "eyJuZXh0VG9rZW4iOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAxMH0="
}

My goal is to chain with jq to extract information and the next page token:

aws batch list-jobs --job-queue $QUEUE --job-status $JOB_STATUS --max-items 1000 | jq -r '.NextToken'

The key is named inconsistently which prevents me from doing that when using the flag --no-paginate. When the --no-paginate flag is passed the key is nextToken and when the --max-items flag (or anything else) is passed then the key is named NextToken.

Reproduction Steps

Set up a job queue and submit some batch jobs. Then run the above commands.

Possible Solution

Capitalise all NextToken for consistency.

Additional Information/Context

No response

CLI version used

aws-cli/2.7.25 Python/3.9.11 Linux/5.4.0-1081-aws exe/x86_64.ubuntu.18 prompt/off

Environment details (OS name and version, etc.)

Ubuntu Server 18.04 LTS (HVM)

[tim-finnigan]

Hi @dereckdemezquita thanks for reaching out. The ListJobs API returns nextToken, but when using --max-items with the CLI pagination it will return NextToken. For more explanation on the --no-paginate behavior you can refer to this comment:

So in general for AWS API's, they will return a next token to indicate if there are more results to return. If you call --no-paginate, it will return the response from a single API request, which will have a next token if not all of the resources are returned in that single response. The CLI by default (and possible) will make the subsequent API calls for you and since it continues paginating until there are no more resource left to list, it will not include a next token in the response displayed because it listed through all of the resources.

[tim-finnigan]

Since we haven't heard back in a few weeks I'm going to close this issue. If you are still affected by this please refer to the comment above and follow up here. Thanks!

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.