apoc icon indicating copy to clipboard operation
apoc copied to clipboard

No results being returned by `apoc.export.json.all` when `writeNodeProperties: true` for a large dataset when writing to a stream

Open elan-sfrancies opened this issue 7 months ago • 1 comments

I am experiencing issues when running the following query against an on-premise docker instance of neo4j Community Edition.

CALL apoc.export.json.all(null, {stream:true, jsonFormat: "JSON_LINES", writeNodeProperties: true})
YIELD file, nodes, relationships, properties, data
RETURN file, nodes, relationships, properties, data

Expected Behavior

The query returns results or displays an error showing why results could not be returned.

Actual Behavior

No results are returned (the following message is seen in the web interface):

(no changes, no records)

How to Reproduce the Problem

Steps

  1. Generate a neo4j instance with ~100,000 nodes, ~100,000 relationships and ~17,000,000 properties
  2. Run the following query:
CALL apoc.export.json.all(null, {stream:true, jsonFormat: "JSON_LINES", writeNodeProperties: true})
YIELD file, nodes, relationships, properties, data
RETURN file, nodes, relationships, properties, data
  1. Observe that results are not returned I have tested this behaviour using the .Net Driver as well as the browser interface.

Screenshots

The results when writeNodeProperties: false:

WriteNodePropertiesFalse

The lack of results when writeNodeProperties: true:

WriteNodePropertiesTrue

Specifications

Memory: 30GB (it appears that the memory use increases during the query before topping out at around 9.5GB and then leveling off.) CPU: 20

Versions

  • OS: Docker for Windows (WSL) on Windows 10
  • Neo4j: neo4j:5.20-community-bullseye
  • Neo4j-Apoc: "NEO4J_PLUGINS=["apoc"]" (latest)

elan-sfrancies avatar Jul 22 '24 09:07 elan-sfrancies