getsentry
Update (Billing) Consumers to emit accepted outcomes for indexed transactions.
After #3662
- Update billing consumer to not emit accepted outcomes for transaction metrics with the
sampled: truetag - Update transaction consumer to not only emit accepted outcomes for the
TransactionsIndexedcategory but also theTransactionscategory.
This will make our counts overall more consistent.
Todo: investigate the same for Spans
These changes should be deployed together, but even if they are, there might be some wrong reporting because consumers do not restart at exactly the same time. I.e. maybe this new behavior should be controlled by a global option.
Let's discuss priority again in Monday's meeting. As we saw for profiles, after metrics extraction the "total" data category (e.g. Transaction) is currently double-represented after metrics extraction, so it is possible to report multiple outcomes for the same payload if it gets dropped after metrics extraction:
flowchart LR
o -->|payload| metrics_extraction
metrics_extraction -->|payload| more_processing
more_processing -->|outcome: dropped| outcomes
metrics_extraction -->|metrics| metrics_consumer
metrics_consumer -->|outcome: accepted| outcomes
We need to either implement this ticket to give the payload full ownership over the data category, or introduce a if metrics_extracted condition to the index_category() function to make sure the payload stops representing the "total" category after metrics extraction.
Decision: implement ownership in consumer as the description states. Requires tagging of the usage metric with a sampled tag.
Idea: Pass the sampled state into metrics extraction by having a composite impl Getter that combines the event with the dynamic sampling decision.
We cannot use the sampling decision to tag the metric anymore. With https://github.com/getsentry/relay/pull/4106 and better enforcement of indexed rate limits, we drop the payload but we still need to keep the transaction counts correct.
After team discussion:
If we update the sentry consumers to take full ownership of the "total" category, we have the problem that payloads dropped sentry-side will no longer show up in the "total" category which is used for billing, even though their metrics are stored and thus create cost / value.
I.e. it's better to keep the model of "metrics represent total category once they exist". If we want to fix the double reporting, we should do it in Relay, potentially by reintroducing the metrics_extracted item header.