incubator-hugegraph
incubator-hugegraph copied to clipboard
apache
fix: tinkerpop unit test
Codecov Report
Attention: Patch coverage is 56.66667% with 13 lines in your changes are missing coverage. Please review.
Project coverage is 26.36%. Comparing base (
7635c67) to head (5d2f23e). Report is 15 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #2381 +/- ##
=============================================
- Coverage 65.59% 26.36% -39.24%
- Complexity 587 3011 +2424
=============================================
Files 511 489 -22
Lines 42590 40102 -2488
Branches 5941 5494 -447
=============================================
- Hits 27936 10571 -17365
- Misses 11840 27945 +16105
+ Partials 2814 1586 -1228
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Currently only 1 failed test found: @javeme any idea about it?
Error: Failures:
Error: EventStrategyProcessTest.shouldTriggerAddVertexWithPropertyThenPropertyAdded:215 expected:<1> but was:<0>
shouldTriggerAddVertexWithPropertyThenPropertyAdded(org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.EventStrategyProcessTest) Time elapsed: 1.461 s <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
at org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.EventStrategyProcessTest.shouldTriggerAddVertexWithPropertyThenPropertyAdded(EventStrategyProcessTest.java:215)
rocksdb 单测问题，看着是我们 rocksdb 不太稳定 Caused by: org.apache.hugegraph.exception.ConnectionException: Failed to open RocksDB 'rocksdb-data/m' Caused by: org.rocksdb.RocksDBException: lock hold by current process, acquire time 1702457768 acquiring thread 140088626837248: rocksdb-data/m/LOCK: No locks available
hbase 有一个单测试是这个问题，close 的时候，有事务没 close，这个貌似以前讨论过 Error: shouldPersistDataOnClose(org.apache.tinkerpop.gremlin.structure.GraphTest) Time elapsed: 2.942 s <<< ERROR! java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'hugegraph_structure_standard' at org.apache.tinkerpop.gremlin.structure.GraphTest.shouldPersistDataOnClose(GraphTest.java:1241)
hbase 有一个单测试是这个问题，close 的时候，有事务没 close，这个貌似以前讨论过 Error: shouldPersistDataOnClose(org.apache.tinkerpop.gremlin.structure.GraphTest) Time elapsed: 2.942 s <<< ERROR! java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'hugegraph_structure_standard' at org.apache.tinkerpop.gremlin.structure.GraphTest.shouldPersistDataOnClose(GraphTest.java:1241)
我加了一点debug信息了，可以根据错误日志一起再看看问题
hbase 有一个单测试是这个问题，close 的时候，有事务没 close，这个貌似以前讨论过 Error: shouldPersistDataOnClose(org.apache.tinkerpop.gremlin.structure.GraphTest) Time elapsed: 2.942 s <<< ERROR! java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'hugegraph_structure_standard' at org.apache.tinkerpop.gremlin.structure.GraphTest.shouldPersistDataOnClose(GraphTest.java:1241)
我加了一点debug信息了，可以根据错误日志一起再看看问题
debug 日志如下：看这些日志，close 抛异常，似乎是符合预期的
Error: -14 07:21:46 [main] [ERROR] o.a.h.StandardHugeGraph - Tx '40' is not closed in thread
java.lang.Exception: null
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.setOpened(StandardHugeGraph.java:1457) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.doOpen(StandardHugeGraph.java:1402) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:95) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:213) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:210) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.AbstractThreadLocalTransaction.doReadWrite(AbstractThreadLocalTransaction.java:92) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.readWrite(AbstractTransaction.java:136) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.hugegraph.StandardHugeGraph.graphTransaction(StandardHugeGraph.java:541) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph.vertices(StandardHugeGraph.java:685) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraph.vertices(TestGraph.java:215) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.TransactionTest$3.run(TransactionTest.java:506) ~[gremlin-test-3.5.1.jar:3.5.1]
Error: -14 07:21:46 [main] [ERROR] o.a.h.t.TestGraphProvider - Error while closing graph 'standardhugegraph[hugegraph_structure_g1]'
java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'hugegraph_structure_g1'
at com.google.common.base.Preconditions.checkState(Preconditions.java:532) ~[guava-30.0-jre.jar:?]
at org.apache.hugegraph.util.E.checkState(E.java:64) ~[hugegraph-common-1.2.0.jar:1.2.0]
at org.apache.hugegraph.StandardHugeGraph.close(StandardHugeGraph.java:985) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraph.close(TestGraph.java:232) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraphProvider.clear(TestGraphProvider.java:361) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.StructureBasicSuite$1.evaluate(StructureBasicSuite.java:115) ~[classes/:?]
at org.apache.tinkerpop.gremlin.AbstractGremlinSuite$1.evaluate(AbstractGremlinSuite.java:235) ~[gremlin-test-3.5.1.jar:3.5.1]
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) ~[junit-4.13.1.jar:4.13.1]
at org.junit.runners.ParentRunner.run(ParentRunner.java:413) ~[junit-4.13.1.jar:4.13.1]
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386) ~[surefire-booter-2.20.jar:2.20]
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323) ~[surefire-booter-2.20.jar:2.20]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143) ~[surefire-booter-2.20.jar:2.20]
2023-12-14 07:21:46 [main] [INFO] o.a.h.StandardHugeGraph - Close graph standardhugegraph[hugegraph_structure_standard]
2023-12-14 07:21:46 [main] [INFO] o.a.h.t.ServerInfoManager - Remove server info: server-tinkerpop
Error: -14 07:21:46 [main] [ERROR] o.a.h.StandardHugeGraph - Tx '48' is not closed in thread
java.lang.Exception: null
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.setOpened(StandardHugeGraph.java:1457) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.doOpen(StandardHugeGraph.java:1402) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:95) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:213) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:210) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.AbstractThreadLocalTransaction.doReadWrite(AbstractThreadLocalTransaction.java:92) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.readWrite(AbstractTransaction.java:136) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.hugegraph.StandardHugeGraph.graphTransaction(StandardHugeGraph.java:541) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph.vertices(StandardHugeGraph.java:685) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraph.vertices(TestGraph.java:215) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.TransactionTest$4.run(TransactionTest.java:545) ~[gremlin-test-3.5.1.jar:3.5.1]
Error: -14 07:21:46 [main] [ERROR] o.a.h.StandardHugeGraph - Tx '40' is not closed in thread
java.lang.Exception: null
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.setOpened(StandardHugeGraph.java:1457) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.doOpen(StandardHugeGraph.java:1402) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:95) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:213) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:210) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.AbstractThreadLocalTransaction.doReadWrite(AbstractThreadLocalTransaction.java:92) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.readWrite(AbstractTransaction.java:136) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.hugegraph.StandardHugeGraph.graphTransaction(StandardHugeGraph.java:541) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph.vertices(StandardHugeGraph.java:685) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraph.vertices(TestGraph.java:215) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.TransactionTest$3.run(TransactionTest.java:503) ~[gremlin-test-3.5.1.jar:3.5.1]
Error: -14 07:21:46 [main] [ERROR] o.a.h.StandardHugeGraph - Tx '51' is not closed in thread
java.lang.Exception: null
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.setOpened(StandardHugeGraph.java:1457) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.doOpen(StandardHugeGraph.java:1402) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:95) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.TransactionTest.lambda$shouldAllowReferenceOfVertexIdOutsideOfOriginalThreadManual$10(TransactionTest.java:827) ~[gremlin-test-3.5.1.jar:3.5.1]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_392]
Error: -14 07:21:46 [main] [ERROR] o.a.h.StandardHugeGraph - Tx '43' is not closed in thread
java.lang.Exception: null
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.setOpened(StandardHugeGraph.java:1457) ~[classes/:?]
at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.doOpen(StandardHugeGraph.java:1402) ~[classes/:?]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:95) ~[gremlin-core-3.5.1.jar:3.5.1]
at org.apache.tinkerpop.gremlin.structure.TransactionTest.lambda$shouldAllowReferenceOfEdgeIdOutsideOfOriginalThreadManual$11(TransactionTest.java:850) ~[gremlin-test-3.5.1.jar:3.5.1]
at java.lang.Thread.run(Thread.java:750) [?:1.8.0_392]
Error: -14 07:21:46 [main] [ERROR] o.a.h.t.TestGraphProvider - Error while closing graph 'standardhugegraph[hugegraph_structure_standard]'
java.lang.IllegalStateException: Ensure tx closed in all threads when closing graph 'hugegraph_structure_standard'
at com.google.common.base.Preconditions.checkState(Preconditions.java:532) ~[guava-30.0-jre.jar:?]
at org.apache.hugegraph.util.E.checkState(E.java:64) ~[hugegraph-common-1.2.0.jar:1.2.0]
at org.apache.hugegraph.StandardHugeGraph.close(StandardHugeGraph.java:985) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraph.close(TestGraph.java:232) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.TestGraphProvider.clear(TestGraphProvider.java:361) ~[classes/:?]
at org.apache.hugegraph.tinkerpop.StructureBasicSuite$1.evaluate(StructureBasicSuite.java:115) ~[classes/:?]
at org.apache.tinkerpop.gremlin.AbstractGremlinSuite$1.evaluate(AbstractGremlinSuite.java:235) ~[gremlin-test-3.5.1.jar:3.5.1]
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) ~[junit-4.13.1.jar:4.13.1]
at org.junit.runners.ParentRunner.run(ParentRunner.java:413) ~[junit-4.13.1.jar:4.13.1]
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) ~[surefire-junit4-2.20.jar:2.20]
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386) ~[surefire-booter-2.20.jar:2.20]
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323) ~[surefire-booter-2.20.jar:2.20]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143) ~[surefire-booter-2.20.jar:2.20]
rocksdb 单测问题，看着是我们 rocksdb 不太稳定 Caused by: org.apache.hugegraph.exception.ConnectionException: Failed to open RocksDB 'rocksdb-data/m' Caused by: org.rocksdb.RocksDBException: lock hold by current process, acquire time 1702457768 acquiring thread 140088626837248: rocksdb-data/m/LOCK: No locks available
could be reproduced in local env: (with same problem)
@javeme seems it failed for a long time (due to the `open RocksDB multi times?) Any known context about it?
CommunityGeneratorTest
我本地尝试将 WAL_PATH, DATA_PATH 按graph 分开，单测就跑过了一部分, 后面仍有类似的问题
I can't run the core test and tinkerpop test locally(unit test is ok), since this commit was introduced:
java.lang.NoClassDefFoundError: org/junit/runner/manipulation/Filter
I can't run the core test and tinkerpop test locally(unit test is ok), since this commit was introduced:
java.lang.NoClassDefFoundError: org/junit/runner/manipulation/Filter
That PR only added a layer of directory structure without making any code level changes, should it be a problem of relative/abs path or environment variable changes?
How does the TP test run in Eclipse? (maybe need update the working space?)
CommunityGeneratorTest
我本地尝试将 WAL_PATH, DATA_PATH 按graph 分开，单测就跑过了一部分, 后面仍有类似的问题
我尝试在 graph close 的时候，强制关闭 rocksdb 实例，这样似乎一类单测问题就通过了，但是考虑一类场景，如果进程意外挂了，graph 没 close，因为文件锁的存在，似乎下次就无法打开了。。这种会有吗？大佬们觉得如何解决比较合适呢？文件锁直接删了？
@Test
public void testOpenGraph() throws Exception {
List<HugeGraph> graphs = openGraphs("test_g1");
final HugeGraph graph = graphs.get(0);
graph.initBackend();
CountDownLatch count = new CountDownLatch(1);
new Thread(()-> {
graph.vertices();
graph.tx().rollback();
count.countDown();
}).start();
count.await();
graph.close();
graphs = openGraphs("test_g1");
HugeGraph graph1 = graphs.get(0);
graph1.initBackend();
graph1.vertices();
}
这个单测可以复线rocksdb 打开的错误，在 setClose 的时候 ref==0 就destoryTransaction 或者 graph.close 的时候或者 forceClose，可以通过
Due to the lack of activity, the current pr is marked as stale and will be closed after 180 days, any update will remove the stale label
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Due to the lack of activity, the current pr is marked as stale and will be closed after 180 days, any update will remove the stale label