Request was blocked due to exceeding usage of resource 'Count' in namespace. Microsoft.TeamFoundation.Framework.Server.RequestBlockedException

[pirate]

We're getting intermittent failing builds in our project: https://github.com/pirate/ArchiveBox/runs/1328177855 due to what looks like rate-limiting.

Is this indeed rate-limiting of some API call, and if so, can we pay to raise the limits somehow?

Cache already exists: Error: reserveCache failed: Cache already exists. Scope: refs/heads/master, Key: layer-docker-layer-caching-Test workflow-a999e9c532683eea4e357c25d5ca0b8962198287270e2697ecd05c39c06e3e6c, Version: 5192f72cb1ac063c20e96d66c297c3edba75ec90ea8ada0c23da6d323a178d87
Stored layer cache: {"key":"layer-docker-layer-caching-Test workflow-a999e9c532683eea4e357c25d5ca0b8962198287270e2697ecd05c39c06e3e6c","cacheId":-1}
Start storing layer cache: {"layerId":"71ef3f3b3628b91e38dc9f125c6b30570b37c8a0a71ea8f5c1c43c97a54fe6c9","key":"layer-docker-layer-caching-Test workflow-71ef3f3b3628b91e38dc9f125c6b30570b37c8a0a71ea8f5c1c43c97a54fe6c9"}
Error: Unexpected error: Error: reserveCache failed: {"$id":"1","innerException":null,"message":"Request was blocked due to exceeding usage of resource 'Count' in namespace ''.","typeName":"Microsoft.TeamFoundation.Framework.Server.RequestBlockedException, Microsoft.TeamFoundation.Framework.Server","typeKey":"RequestBlockedException","errorCode":0,"eventId":3000}
Start storing layer cache: {"layerId":"a1ed3daf83da98e06aa3766fa12fc6d4e2b6cacf2a58cd6609b217734b3698b1","key":"layer-docker-layer-caching-Test workflow-a1ed3daf83da98e06aa3766fa12fc6d4e2b6cacf2a58cd6609b217734b3698b1"}
Error: Unexpected error: Error: reserveCache failed: {"$id":"1","innerException":null,"message":"Request was blocked due to exceeding usage of resource 'Count' in namespace ''.","typeName":"Microsoft.TeamFoundation.Framework.Server.RequestBlockedException, Microsoft.TeamFoundation.Framework.Server","typeKey":"RequestBlockedException","errorCode":0,"eventId":3000}
Start storing layer cache: {"layerId":"e375b26300c04b097ea0c712fdd5746bfbc62f2ca46ad5620f5a88fe2dc2f7e2","key":"layer-docker-layer-caching-Test workflow-e375b26300c04b097ea0c712fdd5746bfbc62f2ca46ad5620f5a88fe2dc2f7e2"}

(this is not urgent, we can live without docker layer caching temporarily)

Thanks!

[marcosh]

got a similar error in my project

Unexpected error: Error: reserveCache failed: {"$id":"1","innerException":null,"message":"Request was blocked due to exceeding usage of resource 'Count' in namespace ''.","typeName":"Microsoft.TeamFoundation.Framework.Server.RequestBlockedException, Microsoft.TeamFoundation.Framework.Server","typeKey":"RequestBlockedException","errorCode":0,"eventId":3000}

[dunglas]

I'm also experimenting this error on a private project.

[toban]

Was seeing this when all our build jobs were sharing the same cache keys. Splitting them so each job had their own key this seemingly stopped occuring.