spark-lucenerdd icon indicating copy to clipboard operation
spark-lucenerdd copied to clipboard

Published 20 hours ago • verified publisher icon zouzias

[lucenerdd] save index to hdfs

Open zouzias opened this issue 5 years ago • 4 comments

WIP

Tries to fix #147

zouzias avatar Mar 14 '19 23:03 zouzias

@zouzias Do you need any support with this?

yeikel avatar Apr 08 '19 01:04 yeikel

Hi @yeikel , if you want to work on the issue feel free to continue the PR.

The missing items are:

  • From every executor copy the index folder to HDFS, i.e., /path_in_hdfs/partition_id/lucene_index.
  • Implement a LuceneRDD constructor that reads the precomputed index from HDFS.

zouzias avatar Apr 09 '19 16:04 zouzias

Any chance you can resolve the merge conflicts here?

yeikel avatar Apr 24 '19 22:04 yeikel

Any chance you can resolve the merge conflicts here?

Done

zouzias avatar Apr 25 '19 09:04 zouzias