BookLore icon indicating copy to clipboard operation
BookLore copied to clipboard
verified publisher icon adityachandelgit

[bug] unicode characters in filename crash booklore

Open feliscat opened this issue 5 months ago • 1 comments 

2025-06-28T18:10:28.988Z  INFO 1 --- [booklore-api] [    virtual-142] c.a.b.s.l.LibraryProcessingService       : Processing file: Integrated Chinese 中文听说读写 Text - Yuehua Liu.pdf
booklore exited with code 0

feliscat avatar Jun 28 '25 18:06 feliscat

Noted, thanks for reporting. I’ll investigate the Unicode filename issue and work on a fix.

acx10 avatar Jun 29 '25 04:06 acx10

There's also an issue when the path contains a \, e.g., this path: .rw-r--r-- 7.7M 18 May 08:03 "/library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U\e (1988)"/'Reconstruction -- eBook.epub' It appears that however this is escaped in the code, it's getting converted to a /. This causes the scanner to stop progressing until the path entry is removed:

2025-08-05T22:23:55.964-04:00  WARN 1 --- [booklore-api] [    virtual-135] c.adityachandel.booklore.util.FileUtils  : Failed to compute hash for file '/library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U/e (1988)/Reconstruction -- eBook.epub': /library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U/e (1988)/Reconstruction -- eBook.epub
2025-08-05T22:23:55.964-04:00  WARN 1 --- [booklore-api] [    virtual-135] c.a.b.s.f.FileProcessingUtils            : Skipping file due to missing hash: /library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U/e (1988)/Reconstruction -- eBook.epub
2025-08-05T22:23:55.970-04:00  WARN 1 --- [booklore-api] [    virtual-135] c.adityachandel.booklore.util.FileUtils  : File does not exist: /library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U/e (1988)/Reconstruction -- eBook.epub
Exception in thread "" java.lang.NullPointerException: Cannot invoke "java.lang.Long.longValue()" because the return value of "com.adityachandel.booklore.util.FileUtils.getFileSizeInKb(java.nio.file.Path)" is null
	at com.adityachandel.booklore.service.BookCreatorService.createShellBook(BookCreatorService.java:28)
	at com.adityachandel.booklore.service.fileprocessor.EpubProcessor.processNewFile(EpubProcessor.java:51)
	at com.adityachandel.booklore.service.fileprocessor.AbstractFileProcessor.processFile(AbstractFileProcessor.java:54)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
	at com.adityachandel.booklore.service.fileprocessor.EpubProcessor$$SpringCGLIB$$0.processFile(<generated>)
	at com.adityachandel.booklore.service.library.FileAsBookProcessor.processLibraryFile(FileAsBookProcessor.java:52)
	at com.adityachandel.booklore.service.library.FileAsBookProcessor.processLibraryFiles(FileAsBookProcessor.java:34)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
	at com.adityachandel.booklore.service.library.FileAsBookProcessor$$SpringCGLIB$$0.processLibraryFiles(<generated>)
	at com.adityachandel.booklore.service.library.LibraryProcessingService.rescanLibrary(LibraryProcessingService.java:58)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
	at com.adityachandel.booklore.service.library.LibraryProcessingService$$SpringCGLIB$$0.rescanLibrary(<generated>)
	at com.adityachandel.booklore.service.library.LibraryService.lambda$rescanLibrary$9(LibraryService.java:178)
	at java.base/java.lang.VirtualThread.run(Unknown Source)



cd 2025-08-05T22:27:37.416-04:00  INFO 1 --- [booklore-api] [     virtual-33] c.a.b.s.w.LibraryFileEventProcessor      : [PROCESS] 'ENTRY_DELETE' event for 'Reconstruction -- eBook.epub'
2025-08-05T22:27:37.421-04:00  INFO 1 --- [booklore-api] [     virtual-33] c.a.b.s.w.LibraryFileEventProcessor      : [FILE_DELETE] '/library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U\e (1988)/Reconstruction -- eBook.epub'
2025-08-05T22:27:37.434-04:00  WARN 1 --- [booklore-api] [     virtual-33] c.a.b.s.w.LibraryFileEventProcessor      : [NOT_FOUND] Book for deleted path '/library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U\e (1988)/Reconstruction -- eBook.epub' not found
2025-08-05T22:27:37.439-04:00  WARN 1 --- [booklore-api] [cTaskExecutor-1] c.a.b.service.monitoring.MonitoringTask  : WatchKey is no longer valid: /library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U\e (1988)
2025-08-05T22:27:37.440-04:00  WARN 1 --- [booklore-api] [cTaskExecutor-1] c.a.b.s.monitoring.MonitoringService     : Removing invalid path from monitoring: /library/books/audiobooks/Foner, Eric/Reconstruction: America's Unfinished Revolution, 1863-1877, U\e (1988)

iconoclasthero avatar Aug 06 '25 02:08 iconoclasthero

I'd also want to open a related bug report that the scanner is shutting down instead of logging and handling errors and continuing on. It happened with an George Kennan article in .pdf I have that evince handles without issue:


2025-08-05T22:28:27.503-04:00  WARN 1 --- [booklore-api] [    virtual-184] c.a.b.s.fileprocessor.PdfProcessor       : Failed to extract PDF metadata for 'The Sources of Soviet Conduct -- Article.pdf': could not execute statement [(conn=574) Data too long for column 'name' at row 1] [insert into author (name) values (?)]; SQL [insert into author (name) values (?)]
Exception in thread "" org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:804)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:758)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:698)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:416)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
	at com.adityachandel.booklore.service.fileprocessor.PdfProcessor$$SpringCGLIB$$0.processFile(<generated>)
	at com.adityachandel.booklore.service.library.FileAsBookProcessor.processLibraryFile(FileAsBookProcessor.java:52)
	at com.adityachandel.booklore.service.library.FileAsBookProcessor.processLibraryFiles(FileAsBookProcessor.java:34)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
	at com.adityachandel.booklore.service.library.FileAsBookProcessor$$SpringCGLIB$$0.processLibraryFiles(<generated>)
	at com.adityachandel.booklore.service.library.LibraryProcessingService.rescanLibrary(LibraryProcessingService.java:58)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
	at com.adityachandel.booklore.service.library.LibraryProcessingService$$SpringCGLIB$$0.rescanLibrary(<generated>)
	at com.adityachandel.booklore.service.library.LibraryService.lambda$rescanLibrary$9(LibraryService.java:178)
	at java.base/java.lang.VirtualThread.run(Unknown Source)

I'm not sure what issue booklore has with this pdf... Data too long for column 'name' at row 1 is apparently enough to cause the scanner to error out and exit? It won't proceed past Ambassador Kennan's article until removed.

iconoclasthero avatar Aug 06 '25 02:08 iconoclasthero