codecs
codecs copied to clipboard
stretchr
Use proper Reader and Writer semantics
Why do you encode/marshal into a freshly allocated []byte, and do you require a []byte for unmarshal? Even proper encodings/csv is hidden behind this memory-hungry, nonscalable interface...
Good question.
We wanted to provide a common interface for all codecs, and since (at the time of initial writing) the JSON encoder used simple marshal/unmarshal methods (i.e. no Encoders which I believe were added later) we went with that.
If you'd like to contribute to designing codecs2, we'd love your input.
On 3 Sep 2013, at 22:11, Tamás Gulácsi [email protected] wrote:
Why do you encode/marshal into a freshly allocated []byte, and do you require a []byte for unmarshal? Even proper encodings/csv is hidden behind this memory-hungry, nonscalable interface...
— Reply to this email directly or view it on GitHub.
Thanks for the positive response!
My use case is a niche, but I have to send unlimited data (records from database query resultset) to the client. I use a hand-made encoder which marshals the result records one after another, and writes them in sequence - thus never more is in memory than one marshaled record.
The generalization is a little bit harder: how should we represent such an iterator-like interface? Maybe we could define a
type Iterable interface { Next() interface{} }
and specialize the marshaler for it - but first a real-world use case would be needed.
The usage of io.Reader / io.Writer is easier, though.
Even without the io.Reader and io.Writer interface, you could use codecs, and just write each response out like you do with your hand-made encoder. At least that way, you could still get the benefit of multiple formats.
An interesting problem.
On 3 Sep 2013, at 23:36, Tamás Gulácsi [email protected] wrote:
Thanks for the positive response!
My use case is a niche, but I have to send unlimited data (records from database query resultset) to the client. I use a hand-made encoder which marshals the result records one after another, and writes them in sequence - thus never more is in memory than one marshaled record.
The generalization is a little bit harder: how should we represent such an iterator-like interface? Maybe we could define a
type Iterable interface { Next() interface{} }
and specialize the marshaler for it - but first a real-world use case would be needed.
The usage of io.Reader / io.Writer is easier, though.
— Reply to this email directly or view it on GitHub.
@tgulacsi @matryer has this issue been resolved? Does it need further discussion? I would like to close it if possible.
I think it should go to designing codecs2 - streaming is definitely a better interface.
On Feb 5, 2014, at 11:31 AM, Tyler [email protected] wrote:
@tgulacsi @matryer has this issue been resolved? Does it need further discussion? I would like to close it if possible.
— Reply to this email directly or view it on GitHub.