orientdb
orientdb copied to clipboard
orientechnologies
OrientDB live lock thread in collection.size
OrientDB Version: 3.0.34
Java Version: 1.8
OS: Linux
Expected behavior
Threads don't live lock and consume all cpu
Actual behavior
This thread is live locked consuming 100% CPU with this stack. I've also attached the full thread dump orient-server-boot.txt
"OrientDB (/172.25.1.18:2424) <- BinaryClient (/172.25.5.171:54870)" #1663 daemon prio=5 os_prio=0 tid=0x00007f8de40a4800 nid=0x53e8 runnable [0x00007f8dee3dd000]
java.lang.Thread.State: RUNNABLE
at java.util.Collections$UnmodifiableCollection.size(Collections.java:1032)
at com.orientechnologies.orient.core.tx.OTransactionRealAbstract.isIndexKeyMayDependOnRid(OTransactionRealAbstract.java:503)
at com.orientechnologies.orient.core.tx.OTransactionRealAbstract.updateIdentityAfterCommit(OTransactionRealAbstract.java:338)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2651)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2489)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.internalCommit(ODatabaseDocumentAbstract.java:2757)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:533)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:103)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:2238)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:2208)
at com.orientechnologies.orient.core.sql.parser.OCommitStatement.executeSimple(OCommitStatement.java:30)
at com.orientechnologies.orient.core.sql.executor.OSingleOpExecutionPlan.executeInternal(OSingleOpExecutionPlan.java:99)
at com.orientechnologies.orient.core.sql.executor.ScriptLineStep.syncPull(ScriptLineStep.java:37)
at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.doExecute(OScriptExecutionPlan.java:89)
at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.fetchNext(OScriptExecutionPlan.java:50)
at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.fetchNext(OLocalResultSet.java:39)
at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.
Steps to reproduce
I'm upserting in OrientDB using multiple threads with remote binary connections.
Hi,
now I'm seeing this behavior with this thread, in this case I'm not sure if it's caused by a query or by a defect. How can I identify the query that is running this?
Thanks
"OrientDB (/172.25.1.18:2424) <- BinaryClient (/172.25.5.106:48914)" #2025 daemon prio=5 os_prio=0 tid=0x00007fd0d4292800 nid=0x73fe runnable [0x00007fcd6be70000]
java.lang.Thread.State: RUNNABLE
at com.orientechnologies.orient.core.tx.OTransactionRealAbstract.isIndexKeyMayDependOnRid(OTransactionRealAbstract.java:517)
at com.orientechnologies.orient.core.tx.OTransactionRealAbstract.isIndexKeyMayDependOnRid(OTransactionRealAbstract.java:504)
at com.orientechnologies.orient.core.tx.OTransactionRealAbstract.updateIdentityAfterCommit(OTransactionRealAbstract.java:338)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2651)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2489)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.internalCommit(ODatabaseDocumentAbstract.java:2757)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:533)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:103)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:2238)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.commit(ODatabaseDocumentAbstract.java:2208)
at com.orientechnologies.orient.core.sql.parser.OCommitStatement.executeSimple(OCommitStatement.java:30)
at com.orientechnologies.orient.core.sql.executor.OSingleOpExecutionPlan.executeInternal(OSingleOpExecutionPlan.java:99)
at com.orientechnologies.orient.core.sql.executor.ScriptLineStep.syncPull(ScriptLineStep.java:37)
at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.doExecute(OScriptExecutionPlan.java:89)
at com.orientechnologies.orient.core.sql.executor.OScriptExecutionPlan.fetchNext(OScriptExecutionPlan.java:50)
at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.fetchNext(OLocalResultSet.java:39)
at com.orientechnologies.orient.core.sql.parser.OLocalResultSet.
Hi @luigidellaquila could you help with identification of the query. I am afraid without proper test case it will be hard to resolve given issue.
@laa I agree with you, without a proper reproducer it's really hard to understand what's happening.
The only clear thing here is that the client is executing a batch script, in particular it's running a COMMIT statement in this moment, but we at least need to know what is the content of the transaction
Thanks
Luigi
Hi,
This slowness doesn't happen with other database instances that I have, so I don't think it's related with an specific index or query but with the data that it's in the graph. I have done some analysis of a heap dump. My feeling is that one node with 80.000 incoming relationships and 80.000 outgoing relationships is being constantly updated and that update is very slow.
In my batch script I'm basically upserting "node1" and "node2" and then creating a relationship between these two nodes by this query "CREATE EDGE edgeClassName UPSERT FROM ($s1) TO ($s2);"
I hope this helps, please let me know if I missed anything that could help on reproduce it.
Thanks a lot for the support!