intermittent firestore UNAUTHENTICATED errors

[michaelAtCoalesce]

firebase-admin 10.3..0

starting yesterday afternoon (July 12 ~4PM PST) our app using the firebase-admin sdk started getting the following errors.

Code: 16 Message: 16 UNAUTHENTICATED: Request had invalid authentication credentials

the issues resolved as of this morning at ~10AM PST.

Filing this ticket to let the firestore team know. While the issue has resolved itself, there were no notices on the firestore status dashboard or any indication that anything failed.

We changed nothing this morning and the firestore SDK went back to working.

[damian4891]

We also have the same problem, but the issue still remains. As we described on stackoverflow, the UNAUTHENTICATED errors occur about 40% of the time. We run 7 different Firebase servers in different parts of the world. All servers are running the exact same code and are configured identically. 2 of them have this issue (australia-southeast1 and us-east1), and 5 didn't. These are production servers supporting over 30,000 users. We had no choice but to create 2 new Firebase servers and divert users to the new servers. This "resolved" the issue.

[cherylEnkidu]

Hi @michaelAtCoalesce @damian4891 ,

Thanks for reporting this. It is a known incident and we have resolved it. Please let us know if you are still experiencing this.