Neo4j migration from 5.5.0 to 5.11.0 - Failed to invoke procedure `apoc.cypher.mapParallel2`: Caused by:RuntimeException: Error polling, timeout

[alexanoid]

trafficstars

I'm trying to migrate from Neo4j 5.5.0 to 5.11.0 (also tried 5.10.0)

The following query works fine on Neo4j 5.5.0:

MATCH  ( childD:Vacancy )  WITH collect({`childD`: childD}) as childDDgRdgd  
CALL apoc.cypher.mapParallel2(" WITH _.childD as childD 
WHERE  ( coalesce(size(childD.employmentIds), 0) <= 0  
OR  exists { MATCH (childD)-[:EMPLOYMENT_AS]-(req0:Requirable) 
WHERE req0.id IN $reqParam0}  )  AND  ( coalesce(size(childD.locationIds), 0) <= 0  
OR  exists { MATCH (childD)-[:LOCATED_IN]-(req1:Requirable) 
WHERE req1.id IN $reqParam1}  )  AND  (childD.`status` = $status5)  
AND  (childD.`active` = $active4)  AND ( (childD.`hourlyRateUsd` >= $hourlyRateUsd2)  
OR  (childD.`salaryUsd` >= $salaryUsd3) ) 
WITH childD RETURN childD  ", {`reqParam1`:[6, 7], `reqParam0`:[26], `active4`:true, `salaryUsd3`:13088, `hourlyRateUsd2`:96, `status5`:'APPROVED'}, childDDgRdgd, 6, 100) YIELD value as value   
WITH value.childD as childD  
return childD

But fails on Neo4j 5.11.0 with:

Failed to invoke procedure `apoc.cypher.mapParallel2`: Caused by: java.lang.RuntimeException: Error polling, timeout of 10 seconds reached.

I also tried to extend the timeout to 100, but with the same result.

Has anything changed between Neo4j 5.5.0 and Neo4j 5.11.0? Maybe some additional configurations are required or something like that?

For my tests, I'm running Neo4j in Docker with:

<NEO4J_db_memory_transaction_total_max>0</NEO4J_db_memory_transaction_total_max> <NEO4J_server_memory_pagecache_size>4G</NEO4J_server_memory_pagecache_size> <NEO4J_server_memory_heap_initial__size>5G</NEO4J_server_memory_heap_initial__size> <NEO4J_server_memory_heap_max__size>5G</NEO4J_server_memory_heap_max__size>

Please help, thanks!

[alexanoid]

I also faced the following one https://stackoverflow.com/questions/76975204/neo4j-access-to-record-went-out-of-bounds-of-the-page-migration-neo4j-5-5-0-to

org.neo4j.driver.exceptions.DatabaseException: Access to record Node[79841,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light,fixedRefs=false] went out of bounds of the page. The record size is 15 bytes, and the access was at offset 1875 bytes into page 146, and the pages have a capacity of 8192 bytes. The mapped store file in question is /data/databases/neo4j/neostore.nodestore.db
at org.neo4j.driver.internal.util.Futures.blockingGet(Futures.java:110) ~[neo4j-java-driver-5.11.0.jar:5.11.0-ea56edc8c126b032c4a7dd8c4c571ec35419ca44]
at org.neo4j.driver.internal.InternalResult.blockingGet(InternalResult.java:111) ~[neo4j-java-driver-5.11.0.jar:5.11.0-ea56edc8c126b032c4a7dd8c4c571ec35419ca44]
at org.neo4j.driver.internal.InternalResult.list(InternalResult.java:87) ~[neo4j-java-driver-5.11.0.jar:5.11.0-ea56edc8c126b032c4a7dd8c4c571ec35419ca44]
at org.neo4j.ogm.drivers.bolt.response.RestModelResponse.<init>(RestModelResponse.java:49) ~[neo4j-ogm-bolt-driver-4.0.6.jar:4.0.6]
at org.neo4j.ogm.drivers.bolt.request.BoltRequest.execute(BoltRequest.java:169) ~[neo4j-ogm-bolt-driver-4.0.6.jar:4.0.6]
at org.neo4j.ogm.session.delegates.ExecuteQueriesDelegate.lambda$query$0(ExecuteQueriesDelegate.java:147) ~[neo4j-ogm-core-4.0.6.jar:4.0.6]
at org.neo4j.ogm.session.Neo4jSession.doInTransaction(Neo4jSession.java:600) ~[neo4j-ogm-core-4.0.6.jar:4.0.6]
at org.neo4j.ogm.session.Neo4jSession.doInTransaction(Neo4jSession.java:574) ~[neo4j-ogm-core-4.0.6.jar:4.0.6]
at org.neo4j.ogm.session.delegates.ExecuteQueriesDelegate.query(ExecuteQueriesDelegate.java:145) ~[neo4j-ogm-core-4.0.6.jar:4.0.6]
at org.neo4j.ogm.session.Neo4jSession.query(Neo4jSession.java:445) ~[neo4j-ogm-core-4.0.6.jar:4.0.6]

[alexanoid]

The same issue is still present with Neo4j 5.12.0

[alexanoid]

Steps to reproduce:

Please start the following database with Neo4j 5.12.0 Community Edition. I run it in Docker.

data.zip

database username: neo4j database password: neo4j12345

ans execute the following query:

MATCH ( childD:Profile ) WITH collect({childD: childD}) as childDDgRdgd CALL apoc.cypher.mapParallel2(" WITH _.childD as childD WHERE (childD.active = $active0) AND (childD.minInfoProvided = $minInfoProvided1) AND (childD.status = $status2) WITH childD RETURN childD ", {active0:true, status2:'APPROVED', minInfoProvided1:true}, childDDgRdgd, 6, 10) YIELD value as value WITH value.childD as childD WITH childD WITH collect({childD:childD }) as aggregate WITH aggregate, size(aggregate) as count UNWIND aggregate as item WITH count, item.childD as childD ORDER BY childD.createdAt DESC SKIP 0 LIMIT 5 OPTIONAL MATCH (childD)-[ru:CREATED_BY]->(u:User) RETURN count, childD AS decision

You should receive:

Failed to invoke procedure apoc.cypher.mapParallel2: Caused by: java.lang.RuntimeException: Error polling, timeout of 10 seconds reached.

If not, reexecute the query until this error occurs.

Regarding:

org.neo4j.driver.exceptions.DatabaseException: Access to record Node[1648,used=false,created=false,rel=-1,prop=-1,labels=Inline(0x0:[]),light,fixedRefs=false] went out of bounds of the page. The record size is 15 bytes, and the access was at offset 150 bytes into page 3, and the pages have a capacity of 8192 bytes. The mapped store file in question is /data/databases/neo4j/neostore.nodestore.db

I don't know how to reliably reproduce this. it occasionally crashes during program execution.

[vga91]

We are investigating it. It seems that the problem is due to some changes made in the 5.9.0 version

[alexanoid]

@vga91 Thank you! Version 5.9.0 is exactly the version in which I first encountered this issue. I have been waiting for new versions in the hope that the problem would be resolved, but unfortunately, it still exists.

This is also related issue I guess: https://github.com/neo4j/neo4j/issues/13315 I never seen such issues on the previous versions

[alexanoid]

Thank you for addressing this. Is there a chance that it will be fixed soon?

[vga91]

Sorry for the late reply, I'll try the solution proposed in the related Neo4j issue Asap

[alexanoid]

@vga91 Thanks for sorting this out! Could you please take a look at this one: https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/3836 ? I noticed some performance degradation.

[vga91]

Ok, I'll take a look asap

[vga91]

Fixed here