Enable Dynamic Sampling

[emmatyping]

We should enable Dynamic Sampling!

Steps to accomplish this:

1. Figure out if we need to add any additional services (c.f https://github.com/getsentry/self-hosted/issues/1860#issuecomment-1351318749)
2. Switch it on in sentry.conf.example.py
3. Check with DS team that changes make sense
4. Note change in release notes

[pavels]

Just a small comment regarding this - https://github.com/getsentry/self-hosted/issues/1835#issuecomment-1349928636 - the breakage observed is with DS disabled. I have never tried to actually enable DS (organizations:dynamic-sampling feature flag).

[Agalin]

I've just enabled dynamic sampling in 22.11. That feature flag is not enough (even in 22.12 which removes server-side-sampling feature flag that's also necessary in 22.11).

Dynamic sampling needs a Quota instance with get_blended_sample_rate implemented and neither base Quota nor RedisQuota implements it. To my understanding this method provides "base" sampling for a project.

If it returns None (default for available implementations) then dynamic sampling won't generate any rules. Otherwise it'll generate a single uniform rule for 1.0 and a set of dynamic rules based on project settings for smaller values with returned one being used for the catch all rule.

I've added a custom implementation inheriting from RedisQuota that takes sampling rate from project options / org options / settings and it generates expected rules.

[petrprikryl]

Is there any time schedule for this according metrics changes in https://github.com/getsentry/self-hosted/releases/tag/23.9.1?

[hubertdeng123]

We don't have a set timeline for this, no.