arangodb-tinkerpop-provider
arangodb-tinkerpop-provider copied to clipboard
ArangoDB-Community
Cannot search by property over a 30M nodes graph
Hello, I am running this query
g.V().has( "myField", P.within([ 10, 13, 14, 100, 110, 111, 131]))
Over a graph with ~30M of nodes.
The process terminates with an error, here the relevant parts of the stack trace
java.util.concurrent.ExecutionException: com.arangodb.ArangoDBException: Response: 404, Error: 1600 - cursor not found
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
....
Caused by: com.arangodb.ArangoDBException: Response: 404, Error: 1600 - cursor not found
at com.arangodb.internal.util.ResponseUtils.checkError(ResponseUtils.java:53)
at com.arangodb.internal.velocystream.VstCommunication.checkError(VstCommunication.java:146)
at com.arangodb.internal.velocystream.VstCommunicationSync.execute(VstCommunicationSync.java:128)
at com.arangodb.internal.velocystream.VstCommunicationSync.execute(VstCommunicationSync.java:42)
at com.arangodb.internal.velocystream.VstCommunication.execute(VstCommunication.java:129)
at com.arangodb.internal.velocystream.VstProtocol.execute(VstProtocol.java:47)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:71)
at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:53)
at com.arangodb.internal.ArangoDatabaseImpl$1.next(ArangoDatabaseImpl.java:232)
at com.arangodb.internal.cursor.ArangoCursorIterator.next(ArangoCursorIterator.java:69)
at com.arangodb.internal.cursor.ArangoCursorImpl.next(ArangoCursorImpl.java:112)
at com.arangodb.tinkerpop.gremlin.client.ArangoDBIterator.next(ArangoDBIterator.java:52)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:148)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
at org.apache.tinkerpop.gremlin.process.traversal.Traversal.fill(Traversal.java:179)
at org.apache.tinkerpop.gremlin.process.traversal.Traversal.toList(Traversal.java:117)
....
How can I overcome this problem?
Hi, not being an expert in the tinkerpop provider, but you probably need to define a longer TTL for your cursor. Maybe @arcanefoam can hint you how to do that with the tinkerpop provider.
How would you do that normally? I means, server configuration side. Not per-query.
Per-query ttl can be specified setting http://arangodb.github.io/arangodb-java-driver/javadoc-6_7/com/arangodb/model/AqlQueryOptions.html#ttl-java.lang.Integer-
Per-query ttl can be specified setting http://arangodb.github.io/arangodb-java-driver/javadoc-6_7/com/arangodb/model/AqlQueryOptions.html#ttl-java.lang.Integer-
Here is says:
If not set, a server-defined value will be used.
Can I change that?
--query.registry-ttl - https://www.arangodb.com/docs/stable/programs-arangod-options.html
Can this be set globally in the java driver, i.e. use for all queries? Cause I think you can pass custom driver settings to the tinkerpop driver.
On Mon, 28 Sep 2020, 10:32 Wilfried Goesgens, [email protected] wrote:
--query.registry-ttl - https://www.arangodb.com/docs/stable/programs-arangod-options.html
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/66#issuecomment-699894657, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQOU3NXJZJNGDSCOOKBTK3SIBJ2XANCNFSM4R25EFRQ .
Then can M. Lissandri open a bug in the tinkerpop driver? That would help us capture the information provided by Michele and add an option to increase the query timeout.
As always, patches/PR are welcomed!
On Mon, 28 Sep 2020, 11:50 Michele Rastelli, [email protected] wrote:
@arcanefoam https://github.com/arcanefoam the driver does not allow setting it globally.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/66#issuecomment-699932079, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQOU3LSIUCGOCWZG66QI7TSIBTADANCNFSM4R25EFRQ .
Just to be clear, do you want me to open a different issue here tinkerpop-provider or on https://github.com/arangodb/arangodb-java-driver for explicitly setting options through the driver?
I can only talk from the tinkerpop side, so I would say in the tinkerpop provider. If we then see thst for it to work changes are required in the java driver we can open an issue there.
On Mon, 28 Sep 2020, 13:34 M. Lissandrini, [email protected] wrote:
Just to be clear, do you want me to open a different issue here tinkerpop-provider or on https://github.com/arangodb/arangodb-java-driver for explicitly setting options through the driver?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/66#issuecomment-699977820, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQOU3ONIGWY75KV5G3ZK5TSIB7EBANCNFSM4R25EFRQ .
Created an issue: https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/67