ingraph icon indicating copy to clipboard operation
ingraph copied to clipboard

Use APOC CSV loader to generate expected results

Open szarnyasg opened this issue 7 years ago • 3 comments

Our current workflow for generating expected test result looks as follows:

  1. Generate/handcraft the CSV files of the inputs.
  2. Fire up a Neo4j 3.0 instance with the Neo4j shell tools.
  3. Load the data with neo4j-import.
  4. Export the data to a GraphML file with neo4j-shell.
  5. Load it with the org.neo4j.shell.tools.imp.format.graphml.XmlGraphMLReader class of the import tools JAR.

This is far too complicated, so we should simplify this and load the data with Cypher instead. However, this would require us to implement 30+ LOAD CSV queries manually.

I discussed this on the Neo4j users slack and it seems that implementing a stored procedure would be the best way to do this: https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/489

szarnyasg avatar Jul 01 '17 11:07 szarnyasg

The stored procedure is almost ready, but this will only be required by the Social Network Benchmark.

szarnyasg avatar Sep 06 '17 17:09 szarnyasg

However, this would require us to implement 30+ LOAD CSV queries manually.

Okay, almost three month later I realized that we could just use APOC's GraphML export/import, which is a lot simpler.

szarnyasg avatar Sep 27 '17 14:09 szarnyasg

It turned out that GraphMLs are a lot more complicated to maintain (the CSVs are no longer a 'single source of truth'), so we should implement this, especially that https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/489 was resolved this summer.

szarnyasg avatar Oct 11 '18 17:10 szarnyasg