Num Checks Before Scale Down & always-decrease-rw-together Don't Work Together

[WalkerCodeRanger]

We have in our config:

num-read-checks-before-scale-down: 30
num-write-checks-before-scale-down: 30

always-decrease-rw-together: true

allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

These settings don't play well together and make it effectively impossible that it would ever scale down. Here is an excerpt of our log file (I shortened the lines and changed the table name):

2014-11-19 21:06:35,286 - TableName - Consumed read units: 0%
2014-11-19 21:06:35,354 - TableName - Consumed write units: 0%
2014-11-19 21:06:35,413 - TableName - Consumed read units: 0%
2014-11-19 21:06:35,434 - TableName - Read throttle count: 0
2014-11-19 21:06:35,434 - TableName - Reached provisioned reads min limit: 25
2014-11-19 21:06:35,434 - TableName - Consecutive read checks 30/30
2014-11-19 21:06:35,479 - TableName - Consumed write units: 0%
2014-11-19 21:06:35,499 - TableName - Write throttle count: 0
2014-11-19 21:06:35,499 - TableName - Consecutive write checks 27/30
2014-11-19 21:06:35,499 - TableName - Changing provisioning to 25 read units and 10 write units
2014-11-19 21:06:35,555 - TableName - Reads could be decreased, but we are waiting for writes to get lower than the threshold before scaling down
2014-11-19 21:06:35,555 - TableName - No changes to perform

2014-11-19 21:07:47,164 - TableName - Consumed read units: 0%
2014-11-19 21:07:47,202 - TableName - Consumed write units: 0%
2014-11-19 21:07:47,255 - TableName - Consumed read units: 0%
2014-11-19 21:07:47,277 - TableName - Read throttle count: 0
2014-11-19 21:07:47,278 - TableName - Reached provisioned reads min limit: 25
2014-11-19 21:07:47,278 - TableName - Consecutive read checks 1/30
2014-11-19 21:07:47,338 - TableName - Consumed write units: 0%
2014-11-19 21:07:47,356 - TableName - Write throttle count: 0
2014-11-19 21:07:47,356 - TableName - Consecutive write checks 28/30
2014-11-19 21:07:47,356 - TableName - No need to change provisioning

2014-11-19 21:08:58,318 - TableName - Consumed read units: 0%
2014-11-19 21:08:58,350 - TableName - Consumed write units: 0%
2014-11-19 21:08:58,399 - TableName - Consumed read units: 0%
2014-11-19 21:08:58,413 - TableName - Read throttle count: 0
2014-11-19 21:08:58,414 - TableName - Reached provisioned reads min limit: 25
2014-11-19 21:08:58,414 - TableName - Consecutive read checks 2/30
2014-11-19 21:08:58,453 - TableName - Consumed write units: 0%
2014-11-19 21:08:58,484 - TableName - Write throttle count: 0
2014-11-19 21:08:58,485 - TableName - Consecutive write checks 29/30
2014-11-19 21:08:58,485 - TableName - No need to change provisioning

2014-11-19 21:10:08,052 - TableName - Consumed read units: 0%
2014-11-19 21:10:08,081 - TableName - Consumed write units: 0%
2014-11-19 21:10:08,124 - TableName - Consumed read units: 0%
2014-11-19 21:10:08,139 - TableName - Read throttle count: 0
2014-11-19 21:10:08,140 - TableName - Reached provisioned reads min limit: 25
2014-11-19 21:10:08,140 - TableName - Consecutive read checks 3/30
2014-11-19 21:10:08,172 - TableName - Consumed write units: 0%
2014-11-19 21:10:08,187 - TableName - Write throttle count: 0
2014-11-19 21:10:08,188 - TableName - Consecutive write checks 30/30
2014-11-19 21:10:08,188 - TableName - Changing provisioning to 35 read units and 5 write units
2014-11-19 21:10:08,220 - TableName - Writes could be decreased, but we are waiting for reads to get lower than the threshold before scaling down
2014-11-19 21:10:08,220 - TableName - No changes to perform

Notice that when "Consecutive read checks 30/30" is hit, it can't scale down because the writes aren't in a state that is ready to scale. However when "Consecutive write checks 30/30" is hit, the consecutive read checks have already been reset.

I think that the consecutive read checks should not have been reset in these situations because the scale wasn't actually performed.

That makes me wonder if there is also a problem when trying to scale, but can't because the limit of 10 tables in Updating status has been hit. It shouldn't reset consecutive checks then either.

Questions:

1. I find it odd that for each check, the "Consumed read units:" and "Consumed write units:" is listed twice. Once at the beginning, and once part way through. What is the reason for that?
2. It seems to always print "Reached provisioned reads min limit: 25" even when provisioned reads are higher than this. Why? It doesn't always print the equivalent for writes.

[thorstenc]

+1 we have the same problem

[highlyunavailable]

We just ran into this as well.

+1

[fedenusy]

+1