xephon-k icon indicating copy to clipboard operation
xephon-k copied to clipboard

[reboot] Add dev log and roadmap

Open at15 opened this issue 6 years ago • 0 comments

It's been a while since I worked on xephon ... a lot has changed except the fact I never finish the projects I started ... Anyway, even go.ice is start being maintained, so xephon-k should get revived as well. Also there are more tsdb written in go now.

TODO

  • [ ] init log to recap what has been done in xephon-k (good old grad school memories)
  • [ ] start playground to test missed things
    • [ ] consistent hashing
    • [ ] serialize inverted index (was building from scratch every time)
    • [ ] compress inverted index
    • [ ] more general data model, druid, piont
    • [ ] compaction
    • [ ] different transport, grpc was never really supported
    • [ ] tcp based line protocol
    • [ ] different compression (existing, RLE, delta RLE are too simple ...)
    • [ ] consider use tree for aggregation
  • [ ] clean up old codebase, I think most of them are no longer useful in the new code base beyond reference ...
  • [ ] a spec for implementation in other language especially java, c++

Dependencies

  • [ ] reboot awesome-tsdb, it's been dead for even longer .... my ‘I will finish it next week' is still there ...
  • [ ] go.ice, gommon need to change accordingly, with dep it should be much easier using replace directive

A MVP would be

  • [ ] store in memory, inverted index + series
  • [ ] WAL (I still don't think WAL is that useful for large volume similar time series data), but adding WAL is much harder than remove it ... so this time, do things in right order
  • [ ] compaction
  • [ ] use consistent hashing to shard series by meta (tags)

at15 avatar Jan 15 '19 06:01 at15