neosemantics icon indicating copy to clipboard operation
neosemantics copied to clipboard

Some questions about importing a very huge knowledge graph

Open lusuon opened this issue 6 years ago • 0 comments

I used this plugin to import an extraordinary huge Knowledge Graph (over 100 GB in total), and have questions about some implementation details of this plugin.

  1. Are there any possible ways to improve the speed of importation?

    • I tried to import some small(relatively, it still takes 4GB of space) parts of data on the server, and it cost almost 5 days.

    • And I have noticed that time of importation seems won't grow linearly with the size of the data, it will slow down as the time goes.

Since the importation of huge file need so much time,here's the second question.

  1. When the importation was interrupted by some unexpected problems, then we rerun the import command:

    • What would the program do with the previous data that were imported before the interruption? (skipped? replaced? or any other solutions to avoid the conflict?)

lusuon avatar Feb 13 '19 04:02 lusuon