getsentry
Logs: Batch Processor Minimize Data Loss
Description
Comply with the new spec added with this PR to the BatchProcessor develop docs, which minimizes data loss by sending spans in memory in a couple of scenarios. For more info, see the project in Linear.
![linear[bot] avatar](https://avatars.githubusercontent.com/in/20150?v=4 "Published by a pub.dev verified publisher"){kind=small}
@philipphofmann Are we going to have 2 Batch Processors for Hybrid SDKs? One for JavaScript envelopes and other for the Native ones? Or are we going to sync both sides in the native Batch Processor?
If we opt for only one Batch Processor, a nice change would be for [PrivateSentrySDKOnly captureEnvelope:envelope] and InternalSentrySdk.captureEnvelope to redirect the envelopes to the native Batch Processor
@lucas-zimerman, my plan would be that the hybrid SDKs will pass down every log and span to the native batch processors, especially for logs for crashes. Not losing any logs for abnormal process terminations is a bit tricky and I wrote a spec for this here: https://github.com/getsentry/sentry-docs/pull/15274/files
Do you think that would work?
@lucas-zimerman, my plan would be that the hybrid SDKs will pass down every log and span to the native batch processors, especially for logs for crashes. Not losing any logs for abnormal process terminations is a bit tricky and I wrote a spec for this here: https://github.com/getsentry/sentry-docs/pull/15274/files
Do you think that would work?
Sounds good to me! I think, the way it is set, we don't really need to queue logs for 5 seconds and then capture, we could send it straight to the Batch processor queue and the timeout will be applied there, what do you think? Or do we still need the delayed queue for Logs separately?
Blocked by https://github.com/getsentry/sentry-java/issues/4826, https://github.com/getsentry/sentry-cocoa/issues/6478 and maybe https://github.com/getsentry/sentry-javascript/issues/17994
I think, the way it is set, we don't really need to queue logs for 5 seconds and then capture, we could send it straight to the Batch processor queue and the timeout will be applied there, what do you think? Or do we still need the delayed queue for Logs separately?
I would just directly forward it to the native batch processors. It's simpler and works better for crash scenarios.