extensions

extensions copied to clipboard

Bug in Distributed counters extension. The firebase log shows this message :

The request was aborted because there was no available instance. Additional troubleshooting documentation can be found at: https://cloud.google.com/functions/docs/troubleshooting#scalability.

I am concerned about the extra billing caused by the errors.

Hi @haris-wilson

Although a number of updates are currently ready for release on this extension https://github.com/firebase/extensions/pull/817.

This seems like a potential new issue i'll can add this to the project board for further investigation.

Also concerned about the extra billing here. What's going on?

@danReynolds what is the extra billing concern? Can you explain a bit more?

@i14h there is a huge log list being generated for my project which if left unwatched causes increase in billing.

Looking into this issue. Can anyone confirm that this is still happening? If so I'd like to investigate.

From what I understand, if there is no available instance, then we are generating an error log on each onWrite trigger, and this could lead to unnecessary billing. (Correct me if i'm wrong)

hey @cabljac,

I can confirm that in our deployment of the Distributed Counter extension, we're seeing multiple 429 warnings (in the 10s) daily for The request was aborted because there was no available instance. Beyond the increased logging, does this warning mean the function was aborted completely and won't be retried at all? if so, we're missing on a lot of realtime updates to our counters.

Any updates on the issue at hand? We've just added this extension and began receiving these errors and warnings.

In the last 12 hours, my project has logged over 32K warnings (POST 429) with the message: The request was aborted because there was no available instance. Should I manually increase the instance count from 1 or leave it like that? Also also noticed some messages on the controllerCore: Query to find shards to aggregate failed. Error: 10 ABORTED: Aborted due to cross-transaction contention. This occurs when multiple transactions attempt to access the same data, requiring Firestore to abort at least one in order to enforce serializability. I've been facing this issue for months now.

Resuming investigation of this now. I've seen in other contexts The request was aborted because there was no available instance when there aren't enough cloud function instances for the events coming through. I'll attempt to repro first

EDIT: reproduced this, continuing to investigate