pcj
pcj copied to clipboard
Published
20 hours ago •
pmem
pmem
"make tests" failed with NullPointerException
/usr/lib/jvm/jdk1.8.0_231/bin/javac -Xlint:unchecked -XDenableSunApiLintControl -proc:none -d target/test_classes -cp src:target/classes:lib:src:lib/ST-4.0.8.jar: src/test/java/tests/PersistentLinkedQueueTest.java src/test/java/tests/PersistentLinkedListTest.java src/test/java/tests/TestCases.java src/test/java/tests/PersistentSIHashMapTest.java src/test/java/tests/PersistentFPTree2Test.java src/test/java/tests/PersistentArrayTest.java src/test/java/tests/PersistentTestRunner.java src/test/java/tests/ObjectDirectoryTest.java src/test/java/tests/PersistentFPTree1Test.java src/test/java/tests/PersistentHashMapTest.java src/test/java/tests/PersistentTupleTest.java src/test/java/tests/PersistentAtomicReferenceTest.java src/test/java/tests/PersistentSkipListMapTest.java src/test/java/tests/MultithreadTest.java src/test/java/tests/PersistentByteBufferTest.java src/test/java/tests/PersistentArrayListTest.java
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Djava.library.path=/usr/local/lib
/usr/lib/jvm/jdk1.8.0_231/bin/java -ea -cp target/classes:lib:src:lib/ST-4.0.8.jar::target/test_classes:src -Djava.library.path=target/cppbuild tests.PersistentTestRunner
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Djava.library.path=/usr/local/lib
Opening heap... Exception in thread "main" java.lang.ExceptionInInitializerError
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at lib.util.persistent.ClassInfo.initReconstructor(ClassInfo.java:139)
at lib.util.persistent.ClassInfo.<init>(ClassInfo.java:74)
at lib.util.persistent.ClassInfo.init(ClassInfo.java:125)
at lib.xpersistent.XHeap.open(XHeap.java:83)
at tests.PersistentTestRunner.main(PersistentTestRunner.java:32)
Caused by: java.lang.NullPointerException
at lib.util.persistent.ObjectCache.lambda$objectForAddress$2(ObjectCache.java:184)
at lib.util.persistent.Util.synchronizedBlock(Util.java:119)
at lib.util.persistent.ObjectCache.objectForAddress(ObjectCache.java:172)
at lib.util.persistent.ObjectCache.getReference(ObjectCache.java:137)
at lib.util.persistent.ObjectCache.get(ObjectCache.java:120)
at lib.util.persistent.ObjectCache.get(ObjectCache.java:114)
at lib.util.persistent.AbstractPersistentMutableArray.getObject(AbstractPersistentMutableArray.java:85)
at lib.util.persistent.AnyPersistent.getObject(AnyPersistent.java:221)
at lib.util.persistent.AbstractPersistentArray.getObjectElement(AbstractPersistentArray.java:70)
at lib.util.persistent.PersistentArray.get(PersistentArray.java:86)
at lib.util.persistent.PersistentHashMap.getNode(PersistentHashMap.java:427)
at lib.util.persistent.PersistentHashMap.get(PersistentHashMap.java:303)
at lib.util.persistent.ObjectDirectory.get(ObjectDirectory.java:36)
at lib.util.persistent.PersistentArrayList.<clinit>(PersistentArrayList.java:57)
... 7 more
make: *** [Makefile:83: tests] Error 1
relevent code is in ObjectCache.java:
@SuppressWarnings("unchecked")
static <T extends AnyPersistent> Ref<T> objectForAddress(long address, boolean forAdmin) {
final Address addr = new Address(address);
return Util.synchronizedBlock(cacheLock, ()->{
Ref<T> ans = null;
if ((ans = (Ref<T>)cache.get(addr)) != null && ans.get() != null) {
// trace(true, address, "OFA HIT: in OC");
// assert(ans.get() != null);
return ans;
}
else {
T obj = null;
MemoryRegion region = new UncheckedPersistentMemoryRegion(address);
long classInfoAddress = region.getLong(0);
ClassInfo ci = ClassInfo.getClassInfo(classInfoAddress); // ci is Null !!!
ObjectType<T> type = Types.typeForName(ci.className());
obj = AnyPersistent.reconstruct(new ObjectPointer<T>((ObjectType)type, region));
ans = new Ref(obj, forAdmin);
if (!Transaction.addReconstructedObject(address, ans)) cache.put(addr, ans);
}
return ans;
});
}```
On ubuntu 19.
jerry@u19:~/opensources/pcj$ sudo fdisk -l /dev/pmem0
Disk /dev/pmem0: 4 GiB, 4294967296 bytes, 8388608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
jerry@u19:~/opensources/pcj$ mount | grep pmem0
/dev/pmem0 on /mnt/mem type ext4 (rw,relatime,dax)
Hi Jerry could you describe the steps to reproduce this. Do you run 'make tests' with a clean heap? Thanks
@asonje I did clean everything (make clean and rm -rf /mnt/mem/* ) to run 'make tests'. This issue would likely happen when key op count or test time increases. I wrote a BasicBenchMarkTest to demo this. https://github.com/JerryYangSH/pcj/blob/master/src/test/java/tests/BasicBenchmarkTest.java
1.copy BasicBenchmarkTest.java into your codebase, 2. rm -rf /mnt/mem/* && make clean&& make && make tests
Opening heap... Cleaning up heap... Heap opened.
****************BasicBenchmarkTest Tests**********************
*Time Bound Testing PersistentFPTree2 (Single Thread, 100000 keys)
[Timer Bound Test 10s] avg lat:
DELETE : 37 (us)
CREATE : 66 (us)
GET : 30 (us)
UPDATE : 36 (us)
[Timer Bound Test 60s] avg lat:
Exception in thread "Thread-0" java.lang.NullPointerException
at lib.util.persistent.AnyPersistent.getChildAddressIterator(AnyPersistent.java:518)
at lib.util.persistent.AnyPersistent.lambda$deleteReference$10(AnyPersistent.java:474)
at lib.util.persistent.Transaction.lambda$run$2(Transaction.java:169)
at lib.util.persistent.Transaction.run(Transaction.java:120)
at lib.util.persistent.Transaction.run(Transaction.java:88)
at lib.util.persistent.Transaction.run(Transaction.java:169)
at lib.util.persistent.AnyPersistent.deleteReference(AnyPersistent.java:463)
at lib.util.persistent.ObjectCache.lambda$static$1(ObjectCache.java:79)
at java.lang.Thread.run(Thread.java:748)
DELETE : 48 (us)
CREATE : 74 (us)
GET : 229 (us)
UPDATE : 65 (us)
[Timer Bound Test 300s] avg lat:
Exception in thread "Thread-10" java.lang.NullPointerException
at lib.util.persistent.ObjectCache.uncommittedConstruction(ObjectCache.java:218)
at lib.util.persistent.Transaction.addReconstructedObject(Transaction.java:251)
at lib.util.persistent.ObjectCache.lambda$objectForAddress$2(ObjectCache.java:187)
at lib.util.persistent.Util.synchronizedBlock(Util.java:125)
at lib.util.persistent.ObjectCache.objectForAddress(ObjectCache.java:172)
at lib.util.persistent.ObjectCache.getReference(ObjectCache.java:137)
at lib.util.persistent.ObjectCache.get(ObjectCache.java:120)
at lib.util.persistent.ObjectCache.get(ObjectCache.java:114)
at lib.util.persistent.AbstractPersistentMutableArray.getObject(AbstractPersistentMutableArray.java:85)
at lib.util.persistent.AnyPersistent.getObject(AnyPersistent.java:221)
at lib.util.persistent.AbstractPersistentArray.getObjectElement(AbstractPersistentArray.java:70)
at lib.util.persistent.PersistentArray.get(PersistentArray.java:86)
at lib.util.persistent.PersistentFPTree2$PersistentLeaf.getSlot(PersistentFPTree2.java:1722)
at lib.util.persistent.PersistentFPTree2.lambda$splitLeafNodeAndUpdateParent$14(PersistentFPTree2.java:1220)
at lib.util.persistent.Transaction.lambda$run$7(Transaction.java:189)
at lib.util.persistent.Transaction.run(Transaction.java:120)
at lib.util.persistent.Transaction.run(Transaction.java:88)
at lib.util.persistent.Transaction.run(Transaction.java:189)
at lib.util.persistent.PersistentFPTree2.splitLeafNodeAndUpdateParent(PersistentFPTree2.java:1217)
at lib.util.persistent.PersistentFPTree2.splitLeafNodeAndUpdateParent(PersistentFPTree2.java:1210)
at lib.util.persistent.PersistentFPTree2.splitChildAndUpdateParent(PersistentFPTree2.java:1203)
at lib.util.persistent.PersistentFPTree2.doPut(PersistentFPTree2.java:1023)
at lib.util.persistent.PersistentFPTree2.put(PersistentFPTree2.java:991)
at lib.util.persistent.PersistentFPTree2.put(PersistentFPTree2.java:77)
at tests.BasicBenchmarkTest.lambda$testBenchmarkTimeBound$0(BasicBenchmarkTest.java:213)
at tests.BasicBenchmarkTest$TimerTestTask.run(BasicBenchmarkTest.java:257)
at java.lang.Thread.run(Thread.java:748)
And it eventually hungs here.
My setup is as below:
jerry@u19:~/opensources/pcj$ cat config.properties
path=/mnt/mem/persistent_heap
size=2147483648jerry@u19:~/opensources/pcj$ !sudo
sudo fdisk -l /dev/pmem0
Disk /dev/pmem0: 4 GiB, 4225761280 bytes, 8253440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
jerry@u19:~/opensources/pcj$ sudo fdisk -l /dev/pmem0
Disk /dev/pmem0: 4 GiB, 4225761280 bytes, 8253440 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
jerry@u19:~/opensources/pcj$ ls -lh /mnt/mem/
total 2.1G
-rw------- 1 jerry jerry 2.0G Dec 20 02:56 persistent_heap
jerry@u19:~/opensources/pcj$ mount | grep pmem0
/dev/pmem0 on /mnt/mem type ext4 (rw,relatime,dax)