pulsar-io-cloud-storage icon indicating copy to clipboard operation
pulsar-io-cloud-storage copied to clipboard
verified publisher icon streamnative

[BUG] Files generated per topic have fluctuating size instead of `batchSize`

Open alpreu opened this issue 3 years ago • 0 comments

Describe the bug The current sink implementation uses one batchSize summing the number of records of all topics together. This results in fluctuating real 'batch' sizes for each topic.

Expected behavior The batchSize should be the minimum amount of records that are flushed for a topic.

alpreu avatar Nov 15 '22 16:11 alpreu