LMW-tree icon indicating copy to clipboard operation
LMW-tree copied to clipboard

Implement distributed version of streaming parallel EM-tree

Open cmdevries opened this issue 11 years ago • 0 comments

Compress transmission of integer accumulators between machines vectors using https://github.com/lemire/FastPFOR.

Hadoop + HDFS (just get hadoop to hand over the bytes, or use HDFS directly).

ZeroMQ + GlusterFS.

Apache Spark might work well with python bindings for library, https://github.com/apache/spark.

HDFS + Erlang scheduler (gascheduler) + C++ code as a simple TCP server.

cmdevries avatar Oct 11 '14 13:10 cmdevries