cpp-statsd-client icon indicating copy to clipboard operation
cpp-statsd-client copied to clipboard

fix: include incoming message size when checking against batch size

Open kwvg opened this issue 8 months ago • 2 comments

Expected behavior

  • Create StatsdClient with batchsize
  • Push a few messages that are under batchsize, remains in same batch
  • Push a message that would go over batchsize if appended, gets put into a new batch
  • Trigger flush, two batches are created, both under batchsize

Actual behavior

  • Create StatsdClient with batchsize
  • Push a few messages that are under batchsize, remains in same batch
  • Push a message that would go over batchsize if appended, gets put into same batch
  • Flush triggered, one batch is created exceeding batchsize

Steps to reproduce

  • Revert e3bffa355c069b89968877adc8d3933cf27d83d9 and run tests, a hyphen (-) is used to indicate a batch

kwvg avatar Mar 09 '25 08:03 kwvg

@kwvg the documentation https://github.com/vthiery/cpp-statsd-client?tab=readme-ov-file#batching explicitly states that the batch size is not a hard limit (indeed it cant be without quite a bit of annoyance, eg. in the case where the batch size is so small a single message wouldnt fit). so i wouldnt characterize this change as a "fix" in this case. this is merely just more of an attempt to make batch size a hard limit right? but even still, the change here does not make it a hard limit specifically in the example i mentioned above. i also see that you keep the additional memory reservation when adding a new batch, which makes no sense if we are trying to stick to a hard limit.

at any rate, i think we should first discuss the merits of a hard limit vs the practical reality. perhaps we can throw or something if someone wants to make a batch size that is too small to be reasonable so as to avoid the pitfall i mentioned above? at present i dont feel strong about having a hard limit vs a soft limit. the reason why we went with a soft one though in the first place was simply because it was just quite a bit less difficult to adhere to logically :smile:

kevinkreiser avatar Mar 10 '25 00:03 kevinkreiser

linting step is broken we'll have to fix that on our end. you can ignore that for now :smile:

kevinkreiser avatar Mar 10 '25 00:03 kevinkreiser