aws-cli icon indicating copy to clipboard operation
aws-cli copied to clipboard

DynamoDB Consumed Capacity is incorrect when CLI makes multiple requests

Open dastbe opened this issue 7 years ago • 10 comments

Hi!

We've noticed that when we perform a ddb count query from the CLI and have it return consumed capacity, the CLI is internally making multiple requests and aggregating the counts across queries. However, it's not also aggregating the consumed capacity. This leads to some odd results like counting 40,000 4kb records and only using 126.5 capacity units.

Looking into it a bit more, it looks like this is where the issue is (not assuming its as simple as removing ConsumedCapacity from non-aggregate keys =D).

cli version: 1.11.160

dastbe avatar Nov 14 '17 02:11 dastbe

I can confirm this. Marking as a feature request. By marking it as non_aggregrate_keys, the CLI only takes the ConsumedCapacity from one of the responses when it builds the full result. I think it could make sense to keep a running total (i.e. result_key) for it like we do for Count and ScannedCount. The main reason this is not done is because the ConsumedCapacity is a structure and it would be difficult to add together each of the different CapacityUnits values in ConsumedCapacity with the current way the CLI pagination works. If you would like to see the ConsumedCapacity for each operation, I would recommend using --no-paginate. However, the downside there is that you will have to paginate the operation yourself if you want all of the items in your query.

kyleknap avatar Nov 15 '17 15:11 kyleknap

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

ASayre avatar Feb 06 '18 10:02 ASayre

Based on community feedback, we have decided to return feature requests to GitHub issues.

jamesls avatar Apr 06 '18 19:04 jamesls

@jamesls any update on this issue?

jthomerson avatar Jul 13 '20 03:07 jthomerson

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. Because it has been longer than one year since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment to prevent automatic closure, or if the issue is already closed, please feel free to reopen it.

github-actions[bot] avatar Jul 13 '21 04:07 github-actions[bot]

@shaunhickson can this be re-opened? It's still a problem. Tangentially, I don't view it as a feature request - it's a bug. We (the callers of the CLI) have no way of knowing how many requests the CLI made, and have to trust the data coming out of it. But we can't trust this data because it's incorrect.

jthomerson avatar Jul 17 '21 23:07 jthomerson

Hi @jthomerson, glad to re-open this since it was closed as stale!

I think that it's still a feature request. The CLI is performing the pagination as specified by DynamoDB, but due to the complexity of aggregating it has not been implemented. The documentation examples could clarify this behavior better as well.

kdaily avatar Jul 19 '21 20:07 kdaily

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

github-actions[bot] avatar Jul 19 '22 20:07 github-actions[bot]

@kdaily could you please reopen? I added an upvote reaction as the bot requested, but apparently that didn't work as it said it would.

jthomerson avatar Jul 21 '22 21:07 jthomerson

@kdaily could you please reopen? I added an upvote reaction as the bot requested, but apparently that didn't work as it said it would.

jthomerson avatar Jul 21 '22 21:07 jthomerson