Hello, so I decided to throw my entire library of books at it and it was doing well until it crashed, seems it hit a possible duplicate book (likely correct) but the thing is that it fails silently and crashes thus stops processing the rest of the files.
Not sure how the whole schema for this is being done, but I guess either skip the dupe and tell the user or somehow add it and mark it as dupe would be a better experience :)
looking at the logs this is what it gives:
Error while processing file Wizard at large - Terry Brooks.epub, error: could not execute batch [(conn=465) Duplicate entry '600-143' for key 'PRIMARY'] [insert into book_metadata_category_mapping (book_id,category_id) values (?,?)]; SQL [insert into book_metadata_category_mapping (book_id,category_id) values (?,?)]; constraint [PRIMARY]
tests_booklore.1.d4jfvlv9dszt@node02 | Exception in thread "" org.springframework.transaction.UnexpectedRollbackException: Transaction silently rolled back because it has been marked as rollback-only
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:804)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:758)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:698)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:416)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:727)
tests_booklore.1.d4jfvlv9dszt@node02 | at com.adityachandel.booklore.service.fileprocessor.EpubProcessor$$SpringCGLIB$$0.processFile(<generated>)
tests_booklore.1.d4jfvlv9dszt@node02 | at com.adityachandel.booklore.service.LibraryProcessingService.processLibraryFile(LibraryProcessingService.java:139)
tests_booklore.1.d4jfvlv9dszt@node02 | at com.adityachandel.booklore.service.LibraryProcessingService.processLibraryFiles(LibraryProcessingService.java:125)
tests_booklore.1.d4jfvlv9dszt@node02 | at com.adityachandel.booklore.service.LibraryProcessingService.rescanLibrary(LibraryProcessingService.java:106)
tests_booklore.1.d4jfvlv9dszt@node02 | at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
tests_booklore.1.d4jfvlv9dszt@node02 | at java.base/java.lang.reflect.Method.invoke(Unknown Source)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
tests_booklore.1.d4jfvlv9dszt@node02 | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:727)
tests_booklore.1.d4jfvlv9dszt@node02 | at com.adityachandel.booklore.service.LibraryProcessingService$$SpringCGLIB$$0.rescanLibrary(<generated>)
tests_booklore.1.d4jfvlv9dszt@node02 | at com.adityachandel.booklore.service.LibraryService.lambda$rescanLibrary$9(LibraryService.java:182)
tests_booklore.1.d4jfvlv9dszt@node02 | at java.base/java.lang.VirtualThread.run(Unknown Source)
Regards
Fabio Rodrigues
@fabiopbx It seems like a concurrency issue. I’ll try to reproduce the problem.
Can I look into this @adityachandelgit ?
Hello, got a similiar issue when updating the database. Unfortunately, if one file fails, the remaining files are not processed anymore. Here is the stacktrace:`
2025-08-19T06:57:22.657Z WARN 1 --- [booklore-api] [ virtual-107] org.hibernate.orm.jdbc.error : (conn=9) Duplicate entry 'BOOK.epub-2' for key 'unique_file_library'
booklore_server | Exception in thread "" org.springframework.dao.DataIntegrityViolationException: could not execute batch [(conn=9) Duplicate entry 'BOOK.epub-2' for key 'unique_file_library'] [update book set added_on=?,book_type=?,current_hash=?,deleted=?,deleted_at=?,file_name=?,file_size_kb=?,file_sub_path=?,library_id=?,library_path_id=?,metadata_match_score=?,similar_books_json=? where id=?]; SQL [update book set added_on=?,book_type=?,current_hash=?,deleted=?,deleted_at=?,file_name=?,file_size_kb=?,file_sub_path=?,library_id=?,library_path_id=?,metadata_match_score=?,similar_books_json=? where id=?]; constraint [unique_file_library]
booklore_server | at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:294)
booklore_server | at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:256)
booklore_server | at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:241)
booklore_server | at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:566)
booklore_server | at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:795)
booklore_server | at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:758)
booklore_server | at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:698)
booklore_server | at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:416)
booklore_server | at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
booklore_server | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
booklore_server | at com.adityachandel.booklore.service.fileprocessor.EpubProcessor$$SpringCGLIB$$0.processFile()
booklore_server | at com.adityachandel.booklore.service.library.FileAsBookProcessor.processLibraryFile(FileAsBookProcessor.java:52)
booklore_server | at com.adityachandel.booklore.service.library.FileAsBookProcessor.processLibraryFiles(FileAsBookProcessor.java:34)
booklore_server | at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
booklore_server | at java.base/java.lang.reflect.Method.invoke(Unknown Source)
booklore_server | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
booklore_server | at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
booklore_server | at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
booklore_server | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
booklore_server | at com.adityachandel.booklore.service.library.FileAsBookProcessor$$SpringCGLIB$$0.processLibraryFiles()
booklore_server | at com.adityachandel.booklore.service.library.LibraryProcessingService.rescanLibrary(LibraryProcessingService.java:66)
booklore_server | at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
booklore_server | at java.base/java.lang.reflect.Method.invoke(Unknown Source)
booklore_server | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
booklore_server | at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380)
booklore_server | at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
booklore_server | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
booklore_server | at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728)
booklore_server | at com.adityachandel.booklore.service.library.LibraryProcessingService$$SpringCGLIB$$0.rescanLibrary()
booklore_server | at com.adityachandel.booklore.service.library.LibraryService.lambda$rescanLibrary$9(LibraryService.java:178)
booklore_server | at java.base/java.lang.VirtualThread.run(Unknown Source)
booklore_server | Caused by: org.hibernate.exception.ConstraintViolationException: could not execute batch [(conn=9) Duplicate entry 'BOOK.epub-2' for key 'unique_file_library'] [update book set added_on=?,book_type=?,current_hash=?,deleted=?,deleted_at=?,file_name=?,file_size_kb=?,file_sub_path=?,library_id=?,library_path_id=?,metadata_match_score=?,similar_books_json=? where id=?]
booklore_server | at org.hibernate.dialect.MariaDBDialect.lambda$buildSQLExceptionConversionDelegate$1(MariaDBDialect.java:374)
booklore_server | at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:34)
booklore_server | at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:115)
booklore_server | at org.hibernate.engine.jdbc.batch.internal.BatchImpl.lambda$performExecution$1(BatchImpl.java:278)
booklore_server | at org.hibernate.engine.jdbc.mutation.internal.PreparedStatementGroupSingleTable.forEachStatement(PreparedStatementGroupSingleTable.java:63)
booklore_server | at org.hibernate.engine.jdbc.batch.internal.BatchImpl.performExecution(BatchImpl.java:253)
booklore_server | at org.hibernate.engine.jdbc.batch.internal.BatchImpl.execute(BatchImpl.java:232)
booklore_server | at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:191)
booklore_server | at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:676)
booklore_server | at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:513)
booklore_server | at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:378)
booklore_server | at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
booklore_server | at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:140)
booklore_server | at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1443)
booklore_server | at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:488)
booklore_server | at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2321)
booklore_server | at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2029)
booklore_server | at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:394)
booklore_server | at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:167)
booklore_server | at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commitNoRollbackOnly(JdbcResourceLocalTransactionCoordinatorImpl.java:249)
booklore_server | at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:243)
booklore_server | at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:90)
booklore_server | at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:562)
booklore_server | ... 33 more
booklore_server | Caused by: java.sql.BatchUpdateException: (conn=9) Duplicate entry 'BOOK.epub-2' for key 'unique_file_library'
booklore_server | at org.mariadb.jdbc.export.ExceptionFactory.createBatchUpdate(ExceptionFactory.java:221)
booklore_server | at org.mariadb.jdbc.client.impl.StandardClient.executePipeline(StandardClient.java:1144)
booklore_server | at org.mariadb.jdbc.ClientPreparedStatement.executeBatchPipeline(ClientPreparedStatement.java:160)
booklore_server | at org.mariadb.jdbc.ClientPreparedStatement.executeInternalPreparedBatch(ClientPreparedStatement.java:140)
booklore_server | at org.mariadb.jdbc.BasePreparedStatement.executeBatchInternal(BasePreparedStatement.java:1694)
booklore_server | at org.mariadb.jdbc.BasePreparedStatement.executeBatch(BasePreparedStatement.java:1680)
booklore_server | at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:128)
booklore_server | at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
booklore_server | at org.hibernate.engine.jdbc.batch.internal.BatchImpl.lambda$performExecution$1(BatchImpl.java:264)
booklore_server | ... 52 more
booklore_server | Caused by: java.sql.SQLIntegrityConstraintViolationException: (conn=9) Duplicate entry 'BOOK.epub-2' for key 'unique_file_library'
booklore_server | at org.mariadb.jdbc.export.ExceptionFactory.createException(ExceptionFactory.java:297)
booklore_server | at org.mariadb.jdbc.export.ExceptionFactory.create(ExceptionFactory.java:378)
booklore_server | at org.mariadb.jdbc.message.ClientMessage.readPacket(ClientMessage.java:187)
booklore_server | at org.mariadb.jdbc.client.impl.StandardClient.readPacket(StandardClient.java:1379)
booklore_server | at org.mariadb.jdbc.client.impl.StandardClient.readResults(StandardClient.java:1318)
booklore_server | at org.mariadb.jdbc.client.impl.StandardClient.readResponse(StandardClient.java:1237)
booklore_server | at org.mariadb.jdbc.client.impl.StandardClient.executePipeline(StandardClient.java:1077)
booklore_server | ... 59 more
`
adityachandelgit
