mercury
mercury copied to clipboard
Eager mode is too eager
The decision to do "eager mode" bulk transfers is made inline, without any knowledge of what else might need to be serialized. That is, if you list an hg_bulk_t before other data structures in serialization order (and use Mercury's default serialization generators), you can easily overflow the unexpected max size and result to bulk xfers, which defeats the purpose of eager mode. The ideal case is that you know the full non-eager size before making the eager xfer decision, to avoid the bulk transfers, but Mercury isn't set up to know that kind of info up front. The resolution of #79 gives an example.
One workaround is to list the hg_bulk_t last in the structure, with the caveat that even that can fail b/c a checksum is added at the end of each serialized buffer. The example in #79 experienced this case (and happens to correspond to leveldb's default benchmark configuration).
Is there an easy way to calculate the needed (non-eager, eager) sizes and allocate the serialize buffer up front rather than piece-by-piece as is currently done?
One solution would be to do 2 passes over the input arguments... which I don't really like.