protobuf icon indicating copy to clipboard operation
protobuf copied to clipboard

Rethink streams and bytes

Open oubiwann opened this issue 7 years ago • 3 comments

The work done on #1 (removal of as many dependencies as possible) resulted in a pause with regard to these libraries:

  • org.flatland/io
  • gloss

It would be nice to not depend upon these old libraries that seem to be doing an enormous amount of work ... work that seems to be overly complex.

If perfect generality wasn't striven for, what might we end up with? How much more simple might the code be, with fewer deps, and maybe even increased efficiency?

Note that both glass and ordered-collections are abandon-ware at this point in time ...

oubiwann avatar Dec 24 '17 03:12 oubiwann

In general, I feel that the implementation of much of this library is rather heavy-handed -- perhaps not for the time it was written -- though it does feel that way these eight years later. @pyr came up with a very nice approach to working with protocol buffers in Mesos with his mesomatic project. It seems that there might be a good deal to take inspiration from, in that.

oubiwann avatar Dec 24 '17 23:12 oubiwann

It will be easier to work on spikes and implementations for this, once #27 is done (and it's almost there).

oubiwann avatar Dec 27 '17 22:12 oubiwann

With #27 done, we can experiment with a new implementation that doesn't use the gloss lib, etc., just handles the bytes directly. The Meson library experimented with this for supporting the RecordIO format over the wire.

Would be fun to checkout state-of-the-art byte stream handling in pure Clojure ... with no libs.

oubiwann avatar Jan 03 '18 02:01 oubiwann