storehaus icon indicating copy to clipboard operation
storehaus copied to clipboard

Published 20 hours ago • verified publisher icon twitter

StorehausOutputFormat

Open sritchie opened this issue 11 years ago • 2 comments

I'm thinking of a Hadoop output format for generating many Storehaus persistences.

The output format would:

  • accept a number of shards and a shard function,
  • assign a shard to each key,
  • sort by (shard, key)
  • creates N Stores on disk.

Paired with #47, and the VersionedTap in dfs-datastores, each output Store would be a proper VersionedStore.

sritchie avatar Mar 19 '13 07:03 sritchie

Relevant: https://github.com/avibryant/rdb

avibryant avatar Mar 19 '13 16:03 avibryant

Awesome, thanks dude.

sritchie avatar Mar 19 '13 17:03 sritchie