add RecordBatchEncoder::records() which creates a RecordIterator instead of requiring a Vec to fill.

[aovestdipaperino]

[aovestdipaperino]

Let me know if you like this change so I can help you fix the conflicts. Basically creating a streaming model for decoding records to reduce the amount of copying required back and forth.

[rukai]

I'm not really sure what this PR is doing, but it might be easier to read if you resolve the merge conflicts? (merge conflicts also seem to be preventing me from approving CI to run)

[rukai]

fwiw I think that in general using iterators to iterate over the contents of the records is a good way to avoid allocations, but we might need to rewrite most of the current records implementation to properly achieve that.

I'm mostly speaking from memories of a long time ago though. Long ago we were thinking of decoding + encoding records in the proxy I work on, so I investigated faster ways to implement record encoding/decoding. But we ended up treating records as just raw bytes.

[aovestdipaperino]

I finally got a chance to update this PR. It seems pretty useful for when a broker requires to "filter" the records to store (pretty convoluted use case, but my broker implementation does require it)