relay
relay copied to clipboard
getsentry
IPv6 for sentry ingest
Problem Statement
With AWS moving to charging users for all IPv4 addresses (EIPs), not just unused EIPs, extremely-large and extremely-small customers are being financially incentivised to move to IPv6-only networking. Unfortunately, Sentry users can't avoid IPv4 because the sentry ingest endpoints don't have any AAAA records and thus can only be reached over IPv4:
$ dig AAAA o22412.ingest.sentry.io @8.8.8.8
; <<>> DiG 9.10.6 <<>> AAAA o22412.ingest.sentry.io @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8270
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;o22412.ingest.sentry.io. IN AAAA
This is one of those things that sounds like a trivial change ("just assign the loadbalancers ipv6 addresses in the GCP console!") to an outsider but I understand that its scope is obviously larger than that, as everything downstream of that load balancer needs to be able to handle ipv6 addresses as an x-forwarded-for header, ip address in a log entry, etc.
Solution Brainstorm
It would be cool if the infrastructure for your ingest endpoints supported IPv6 so AWS customers could save some money that could then be spent on, for example, their sentry budgets.
Product Area
Unknown
![getsantry[bot] avatar](https://avatars.githubusercontent.com/in/66573?v=4 "Published by a pub.dev verified publisher"){kind=small}
@r4v5 Thanks for opening this issue. As you mentioned above, there can be some non trivial backend changes which has to be tackled before the IPv6 could be enabled.
@Fwang36 , can this be moved to the proper project? This is something what our SRE team have to look into as well, together with sentry backend team I guess.
@tonyo heya, I think you're already aware of this already.
Where can this issue be transferred so the SRE team can better track it?
From the internally linked doc:
We don’t have a set roadmap for this, but we’re planning to launch initial support before the end of the year (2024).
This issue has gone three weeks without activity. In another week, I will close it.
But! If you comment or otherwise update it, I will reset the clock, and if you remove the label Waiting for: Community, I will leave it alone ... forever!
"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀
This seems to be a valid ask, removing "Waiting for: Community" so the bot doesn't close.
OPS update on this:
- we currently have 3 draft PRs to support ipv6 ingestion on our infra, specifically:
- https://github.com/getsentry/ops/pull/10898
- https://github.com/getsentry/ops/pull/10899
- https://github.com/getsentry/ops/pull/10900
- the upper PRs target infra capabilities only
- we still need coordination with product in order to support ipv6 event filtering in Sentry and relevant communications to customers in order to let 'em update their rules
Currently targeted for Q3'24.
Note: we already have an ipv6 ingestion address provisioned but unbound from our DNS records; such address (2600:1901:0:e44a::) can be targeted with host aliases for specific ingestion DSN (o<id>.ingest.us.sentry.io) and the traffic will be ingested already. As mentioned above, the events won't have proper source address information (until the upper PRs are merged) and ipv4 based filters won't work.
Happy birthday to this issue! I understand Q3 '24 was the target, is there an update now that it's 2025? Need to know if this affects my launch planning :/
Hey, any updates about this?
Sorry for the long delay on this. We had some changes in the plan strategy and this went back to the backlog.
The good news is: we're currently working on some big changes on the ingestion infrastructure that should land in a couple of months at max, and proper ipv6 support will be included in such changes.
@rsinger86 @gi0baro has the new clusters setup, but it's not yet in production. We're working on switching over our existing infra to the freshly setup clusters.
I do not have a concrete timeline for you, this is mostly managed by our infra team(s). I am pushing for sooner rather than later though.
A quick status update on this: we're currently beta-testing the new infrastructure, with expected GA by the end of the month on all our regions.