sevenzipjbinding icon indicating copy to clipboard operation
sevenzipjbinding copied to clipboard

Too many open files when compress

Open XuQK opened this issue 2 years ago • 0 comments

Is there thread limit when compress?

I found a error when I use SevenZip.openOutArchiveZip.createArchive to compress thousands small file in my Android:

packageName.utils.filezip.compress.FileCompressor$compressToZip$1$2$1.getStream(FileCompressor.kt:151)
java.io.FileNotFoundException: /storage/emulated/0/Android/data/com.tencent.mobileqq/qcircle/file/download/-1270698121774109711/rocket_single_click_new/single_click_00029.png: open failed: EMFILE (Too many open files)
	at libcore.io.IoBridge.open(IoBridge.java:512)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:274)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:141)
	at packageName.utils.filezip.compress.FileCompressor$compressToZip$1$2$1.getStream(FileCompressor.kt:146)
	at net.sf.sevenzipjbinding.impl.OutArchiveImpl.nativeUpdateItems(Native Method)
	at net.sf.sevenzipjbinding.impl.OutArchiveImpl.doUpdateItems(OutArchiveImpl.java:141)
	at net.sf.sevenzipjbinding.impl.OutArchiveImpl.createArchive(OutArchiveImpl.java:150)
	at net.sf.sevenzipjbinding.impl.OutArchiveZipImpl.createArchive(OutArchiveZipImpl.java:29)
	at packageName.utils.filezip.compress.FileCompressor$compressToZip$1.invokeSuspend(FileCompressor.kt:78)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: android.system.ErrnoException: open failed: EMFILE (Too many open files)
	at libcore.io.Linux.open(Native Method)
	at libcore.io.BlockGuardOs.open(BlockGuardOs.java:207)
	at libcore.io.IoBridge.open(IoBridge.java:498)

I print Thread.currentThread().name in "IOutCreateCallback.setCompleted", then I'm seeing an increasing number of threads like this:

Thread-1
Thread-2
Thread-3
Thread-4
.
.
.
Thread-343314
.
.
.

Is this a threading issue? how do I fix it? Thanks

PS: this my IOutCreateCallback.getStream implement, maybe useful:

override fun getStream(index: Int): ISequentialInStream? {
    return if (File(items[index].pathInFileSystem).isDirectory) {
        null
    } else {
        val inStream: RandomAccessFileInStream
        inStream = RandomAccessFileInStream(
            RandomAccessFile(items[index].pathInFileSystem, "r")
        )
        closableList.add(inStream)
        inStream
    }
}

XuQK avatar May 19 '22 11:05 XuQK