mercury icon indicating copy to clipboard operation
mercury copied to clipboard

Eager mode is too eager

Open JohnPJenkins opened this issue 8 years ago • 1 comments

The decision to do "eager mode" bulk transfers is made inline, without any knowledge of what else might need to be serialized. That is, if you list an hg_bulk_t before other data structures in serialization order (and use Mercury's default serialization generators), you can easily overflow the unexpected max size and result to bulk xfers, which defeats the purpose of eager mode. The ideal case is that you know the full non-eager size before making the eager xfer decision, to avoid the bulk transfers, but Mercury isn't set up to know that kind of info up front. The resolution of #79 gives an example.

One workaround is to list the hg_bulk_t last in the structure, with the caveat that even that can fail b/c a checksum is added at the end of each serialized buffer. The example in #79 experienced this case (and happens to correspond to leveldb's default benchmark configuration).

Is there an easy way to calculate the needed (non-eager, eager) sizes and allocate the serialize buffer up front rather than piece-by-piece as is currently done?

JohnPJenkins avatar Mar 30 '16 18:03 JohnPJenkins

One solution would be to do 2 passes over the input arguments... which I don't really like.

soumagne avatar Mar 30 '16 19:03 soumagne