bookkeeper
bookkeeper copied to clipboard
apache
SortedLedgerStorage > Spammy log on mem-table size limit reached
BUG REPORT
When the Bookie tries to flush the MemEntryTable on an addEntry it started to spam the log "Reached size null".
Describe the bug
20-05-04-00-02-44 Reached size null
java.io.FileNotFoundException: /data/magnews/bookie/./bookie_data/current/73/72/67372.idx (Too many open files)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:345)
at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:259)
at java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:214)
at org.apache.bookkeeper.bookie.FileInfo.readHeader(FileInfo.java:213)
at org.apache.bookkeeper.bookie.FileInfo.checkOpen(FileInfo.java:310)
at org.apache.bookkeeper.bookie.FileInfo.checkOpen(FileInfo.java:278)
at org.apache.bookkeeper.bookie.FileInfo.size(FileInfo.java:388)
at org.apache.bookkeeper.bookie.IndexPersistenceMgr.updatePage(IndexPersistenceMgr.java:642)
at org.apache.bookkeeper.bookie.IndexInMemPageMgr.grabLedgerEntryPage(IndexInMemPageMgr.java:447)
at org.apache.bookkeeper.bookie.IndexInMemPageMgr.getLedgerEntryPage(IndexInMemPageMgr.java:412)
at org.apache.bookkeeper.bookie.IndexInMemPageMgr.putEntryOffset(IndexInMemPageMgr.java:571)
at org.apache.bookkeeper.bookie.LedgerCacheImpl.putEntryOffset(LedgerCacheImpl.java:108)
at org.apache.bookkeeper.bookie.InterleavedLedgerStorage.processEntry(InterleavedLedgerStorage.java:539)
at org.apache.bookkeeper.bookie.SortedLedgerStorage.process(SortedLedgerStorage.java:271)
at org.apache.bookkeeper.bookie.EntryMemTable.flushSnapshot(EntryMemTable.java:256)
at org.apache.bookkeeper.bookie.EntryMemTable.flush(EntryMemTable.java:206)
at org.apache.bookkeeper.bookie.SortedLedgerStorage$1.run(SortedLedgerStorage.java:300)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Expected behavior
I expect that if it can't flush the checkpoint for this type of Exception the service stops
@mino181295 How is the issue comes out? Is there any steps to reproduce this?
We have seen that only once. When we stopped and restarted the Bookie and it worked fine. Actually I can't provide more context.
