self-hosted icon indicating copy to clipboard operation
self-hosted copied to clipboard

Published 20 hours ago • verified publisher icon getsentry

relay_server::services::project_upstream

Open BenJChen opened this issue 1 year ago • 5 comments

Self-Hosted Version

24.7.1

CPU Architecture

x86_64

Docker Version

24.0.6

Docker Compose Version

2.20.3

Steps to Reproduce

After a large number of errors entered the sentry service, sentry crashed

Expected Result

The relay service returns to normal, and you can see error and performance statistics.

Actual Result

Image

Yesterday, due to a large number of errors reported by the project, sentry failed. We tried removing all Kafka-related volumes and recreating them, restarting sentry and other methods. The project can run normally, but the data cannot be written. 12 hours have passed and we still cannot see new data in sentry. The data, nginx view request is normal, but a large number of errors appear in the relay, how to solve this

Event ID

No response

BenJChen avatar Aug 07 '24 06:08 BenJChen

Related: https://github.com/getsentry/relay/issues/3828

What do you mean by "sentry crashed"? Is the web container failing?

joshuarli avatar Aug 12 '24 20:08 joshuarli

Related: getsentry/relay#3828

What do you mean by "sentry crashed"? Is the web container failing?

The error and performance data cannot be updated for more than ten hours. The nginx view request is normal and the relay prompts relay_server::services::project_upstream.

BenJChen avatar Aug 13 '24 01:08 BenJChen

Have you tried restarting Relay? Is it just total failure to ingest any more events?

joshuarli avatar Aug 13 '24 02:08 joshuarli

您是否尝试过重新启动 Relay？是否完全无法接收更多事件？

Have you tried restarting Relay? Is it just total failure to ingest any more events?

Restarting relay and even restarting sentry failed to solve the problem. It was not until I tried to clear all redis caches and restart sentry that relay returned to normal.

BenJChen avatar Aug 13 '24 03:08 BenJChen

Hi, did you experience any overload of Redis when the errors started happening?

iambriccardo avatar Aug 26 '24 08:08 iambriccardo

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

getsantry[bot] avatar Sep 19 '24 07:09 getsantry[bot]