google-cloud-go icon indicating copy to clipboard operation
google-cloud-go copied to clipboard
verified publisher icon googleapis

pubsub: randomly got an unauthorized error in rpc call

Open git-hulk opened this issue 1 year ago • 3 comments

Client

PubSub

Environment

Ubuntu Docker on GKE

$ go 1.21

Additional context

We got the following unauthenticated error with Request had invalid euc header randomly in our production environment. From the error description, it seems this was caused by the PubSub server-side behavior but cannot find any related documentation about this.

rpc error: code = Unauthenticated desc = Request had invalid euc header.

git-hulk avatar Mar 20 '25 06:03 git-hulk

@hongalex wants to know if the euc is the short form for End User Computation?

git-hulk avatar Mar 21 '25 03:03 git-hulk

Hi, this feels like an authentication / GKE issue that is not specific to the Go Pub/Sub library. If you have a support contract, could you file a support ticket so we can look into this further?

if the euc is the short form for End User Computation?

I don't think so, but would be good to clarify this in the ticket you open.

hongalex avatar Mar 25 '25 15:03 hongalex

git-hulk Could you also let us know the time / date of the incident + if there was a recurrence or a one time issue?

mukund-ananthu avatar Mar 25 '25 17:03 mukund-ananthu

Happened to us as well, 17 occurrences in the past 7 days:

  1. 2025-04-28T04:44:38+00:00
  2. 2025-04-28T04:44:38+00:00
  3. 2025-04-28T04:44:38+00:00
  4. 2025-04-28T04:44:36+00:00
  5. 2025-04-28T04:44:36+00:00
  6. 2025-04-28T04:44:36+00:00
  7. 2025-04-28T04:20:04+00:00
  8. 2025-04-28T04:20:04+00:00
  9. 2025-04-28T04:19:33+00:00
  10. 2025-04-28T04:19:33+00:00
  11. 2025-04-28T04:19:25+00:00
  12. 2025-04-28T04:19:25+00:00
  13. 2025-04-28T04:19:25+00:00
  14. 2025-04-28T04:19:25+00:00
  15. 2025-04-26T01:23:02+00:00
  16. 2025-04-24T13:52:54+00:00
  17. 2025-04-24T13:52:48+00:00

agis avatar Apr 28 '25 08:04 agis

This is been happening for us in production for over a month, and it seems to be getting worse, with over half of the occurrences happening in the last seven days. We have not changed our Go client libraries for PubSub.

sgp avatar Apr 29 '25 18:04 sgp

This is a known issue with the backend servers and is not specific to the Go Pub/Sub library or any specific language.

If you have a support ticket, I recommend filing a ticket there so this can tracked and triaged properly.

hongalex avatar May 02 '25 21:05 hongalex

This should be resolved now but if you are still experiencing this, please continue to use support channels (tickets) to address.

hongalex avatar Jul 23 '25 21:07 hongalex