adityachandelgit
only 44 from 805 PDF files listed in a folder
🐛 Bug Report for Booklore
Thank you for helping us improve Booklore! Please fill out the sections below.
📝 What happened?
I added a new library from a folder where 800+ PDF files are present. booklore ended with only 44 PDF ???
🔄 Steps to Reproduce
no error displayed, only 800+ files missing
✅ Expected Behavior
800+ PDF files listed in the new library
📸 Screenshots / Error Messages
💻 Environment
- Version: v1.10.0
- OS: debian 12 AMD64 docker 27 on Ryzen 5700U 32gb RAM 2*1Tb disks
- Browser: brave on win10
- Installation: docker compose
📌 Additional Context
Other libraries verified good
✨ Possible Solution (Optional)
fix "sync library files" task ??
files in CyberSec folder
Library in Booklore
Booklore updated to version 1.12.0
Hi,
Can you provide us with logs or any additional information? For example, is there anything special about the 44 files (or the others that didn't get imported) that got imported, regarding metadata, filenames, etc.? I'm afraid this isn't much to go on. My initial thinking is that it's a race condition because sync files and library creation run in parallel with each other (with the assumption that library creation will complete after sync files). However, I'd like to see if maybe there's a more mundane problem going on.
I ran into the same issue today, when i copied in my books into a folder that i then took as the library (I used default settings). It stopped in my case at only 2 books and never continued. I couldn’t really capture the logs at the time because I was still getting Booklore set up.
Instead I put all files into bookdrop, and after multiple restarts it slowly crawled its way through the files. Now I believe all files are imported correctly.
2025-11-27T18:52:51.211+01:00 ERROR 10 --- [booklore-api] [nPool-worker-46] c.a.b.s.metadata.parser.GoodReadsParser : Invalid publication time: null, Error: For input string: "null"
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
▀█████████▄ ▄██████▄ ▄██████▄ ▄█ ▄█▄ ▄█ ▄██████▄ ▄████████ ▄████████
███ ███ ███ ███ ███ ███ ███ ▄███▀ ███ ███ ███ ███ ███ ███ ███
███ ███ ███ ███ ███ ███ ███▐██▀ ███ ███ ███ ███ ███ ███ █▀
▄███▄▄▄██▀ ███ ███ ███ ███ ▄█████▀ ███ ███ ███ ▄███▄▄▄▄██▀ ▄███▄▄▄
▀▀███▀▀▀██▄ ███ ███ ███ ███ ▀▀█████▄ ███ ███ ███ ▀▀███▀▀▀▀▀ ▀▀███▀▀▀
███ ██▄ ███ ███ ███ ███ ███▐██▄ ███ ███ ███ ▀███████████ ███ █▄
███ ███ ███ ███ ███ ███ ███ ▀███▄ ███▌ ▄ ███ ███ ███ ███ ███ ███
▄█████████▀ ▀██████▀ ▀██████▀ ███ ▀█▀ █████▄▄██ ▀██████▀ ███ ███ ██████████
▀ ▀ ███ ███ (v1.12.0)
Powered By Spring Boot : 3.5.1
2025-11-30T14:24:30.778+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.BookloreApplication : Starting BookloreApplication v0.0.1-SNAPSHOT using Java 25.0.1 with PID 11 (/app/app.jar started by root in /)
2025-11-30T14:24:30.787+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.BookloreApplication : No active profile set, falling back to 1 default profile: "default"
2025-11-30T14:24:36.071+01:00 INFO 11 --- [booklore-api] [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2025-11-30T14:24:36.492+01:00 INFO 11 --- [booklore-api] [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 400 ms. Found 43 JPA repository interfaces.
2025-11-30T14:24:39.209+01:00 INFO 11 --- [booklore-api] [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2025-11-30T14:24:39.425+01:00 INFO 11 --- [booklore-api] [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection org.mariadb.jdbc.Connection@49665f92
2025-11-30T14:24:39.428+01:00 INFO 11 --- [booklore-api] [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2025-11-30T14:24:39.507+01:00 INFO 11 --- [booklore-api] [ main] org.flywaydb.core.FlywayExecutor : Database: jdbc:mariadb://minipc/booklore?user=booklore&password=******** (MariaDB 11.4)
2025-11-30T14:24:39.689+01:00 INFO 11 --- [booklore-api] [ main] o.f.core.internal.command.DbValidate : Successfully validated 63 migrations (execution time 00:00.134s)
2025-11-30T14:24:39.714+01:00 INFO 11 --- [booklore-api] [ main] o.f.core.internal.command.DbMigrate : Current version of schema booklore: 63
2025-11-30T14:24:39.723+01:00 INFO 11 --- [booklore-api] [ main] o.f.core.internal.command.DbMigrate : Schema booklore is up to date. No migration necessary.
2025-11-30T14:24:39.947+01:00 INFO 11 --- [booklore-api] [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2025-11-30T14:24:40.089+01:00 INFO 11 --- [booklore-api] [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 7.1.3.Final
2025-11-30T14:24:41.022+01:00 INFO 11 --- [booklore-api] [ main] o.s.o.j.p.SpringPersistenceUnitInfo : No LoadTimeWeaver setup: ignoring JPA class transformer
2025-11-30T14:24:43.892+01:00 INFO 11 --- [booklore-api] [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000489: No JTA platform available (set 'hibernate.transaction.jta.platform' to enable JTA platform integration)
2025-11-30T14:24:43.895+01:00 INFO 11 --- [booklore-api] [ main] org.hibernate.orm.jdbc.batch : HHH100501: Automatic JDBC statement batching enabled (maximum batch size 500)
2025-11-30T14:24:43.899+01:00 INFO 11 --- [booklore-api] [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2025-11-30T14:24:53.990+01:00 INFO 11 --- [booklore-api] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port 8080 (http)
2025-11-30T14:24:54.009+01:00 INFO 11 --- [booklore-api] [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2025-11-30T14:24:54.010+01:00 INFO 11 --- [booklore-api] [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.42]
2025-11-30T14:24:54.053+01:00 INFO 11 --- [booklore-api] [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2025-11-30T14:24:54.053+01:00 INFO 11 --- [booklore-api] [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 23033 ms
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
2025-11-30T14:24:54.200+01:00 INFO 11 --- [booklore-api] [ main] o.s.d.j.r.query.QueryEnhancerFactory : Hibernate is in classpath; If applicable, HQL parser will be used.
2025-11-30T14:24:56.510+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.f.BookFileProcessorRegistry : Initialized BookFileProcessorRegistry with 3 processors for 3 types
2025-11-30T14:24:56.576+01:00 INFO 11 --- [booklore-api] [ virtual-59] c.a.b.s.w.LibraryFileEventProcessor : LibraryFileEventProcessor virtual thread started.
2025-11-30T14:24:56.606+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.monitoring.MonitoringService : Starting file change processor...
2025-11-30T14:24:56.734+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.monitoring.MonitoringService : Registered 1 folders for library 'CyberSec'
2025-11-30T14:24:56.736+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.monitoring.MonitoringService : Registered 1 folders for library 'CyberSouverain'
2025-11-30T14:24:56.737+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.monitoring.MonitoringService : Registered 1 folders for library 'CyberIA'
2025-11-30T14:24:56.737+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.monitoring.MonitoringService : Registered 31 libraries for recursive monitoring
2025-11-30T14:24:56.737+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.service.library.LibraryService : Monitoring initialized with 31 libraries
2025-11-30T14:24:57.376+01:00 INFO 11 --- [booklore-api] [ main] c.a.b.s.b.BookdropMonitoringService : Starting bookdrop folder monitor: /bookdrop
2025-11-30T14:24:58.886+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.config.WebSocketConfig : WebSocket endpoint registered at /ws
2025-11-30T14:24:59.070+01:00 WARN 11 --- [booklore-api] [ main] JpaBaseConfiguration$JpaWebConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2025-11-30T14:24:59.992+01:00 INFO 11 --- [booklore-api] [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint beneath base path '/actuator'
2025-11-30T14:25:00.866+01:00 INFO 11 --- [booklore-api] [ main] o.s.m.s.b.SimpleBrokerMessageHandler : Starting...
2025-11-30T14:25:00.866+01:00 INFO 11 --- [booklore-api] [ main] o.s.m.s.b.SimpleBrokerMessageHandler : BrokerAvailabilityEvent[available=true, SimpleBrokerMessageHandler [org.springframework.messaging.simp.broker.DefaultSubscriptionRegistry@5db34d24]]
2025-11-30T14:25:00.868+01:00 INFO 11 --- [booklore-api] [ main] o.s.m.s.b.SimpleBrokerMessageHandler : Started.
2025-11-30T14:25:00.886+01:00 INFO 11 --- [booklore-api] [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port 8080 (http) with context path '/'
2025-11-30T14:25:00.908+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.BookloreApplication : Started BookloreApplication in 32.685 seconds (process running for 27.183)
2025-11-30T14:25:01.011+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.config.TaskSchedulerConfig : Application ready, initializing scheduled tasks
2025-11-30T14:25:01.020+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.service.task.TaskService : Initializing 4 scheduled tasks
2025-11-30T14:25:01.049+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.service.task.TaskService : Scheduled task CLEANUP_DELETED_BOOKS with cron expression: 0 40 0 * * 1
2025-11-30T14:25:01.050+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.service.task.TaskService : Scheduled task CLEANUP_TEMP_METADATA with cron expression: 0 45 0 * * 1
2025-11-30T14:25:01.050+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.service.task.TaskService : Scheduled task SYNC_LIBRARY_FILES with cron expression: 0 0 1 * * *
2025-11-30T14:25:01.051+01:00 INFO 11 --- [booklore-api] [ main] c.a.booklore.service.task.TaskService : Scheduled task UPDATE_BOOK_RECOMMENDATIONS with cron expression: 0 30 1 * * *
2025-11-30T14:26:00.830+01:00 INFO 11 --- [booklore-api] [MessageBroker-1] o.s.w.s.c.WebSocketMessageBrokerStats : WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[processed CONNECT(0)-CONNECTED(0)-DISCONNECT(0)], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 6, active threads = 1, queued tasks = 5, completed tasks = 0]
2025-11-30T14:32:46.001+01:00 INFO 11 --- [booklore-api] [nio-8080-exec-2] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2025-11-30T14:32:46.002+01:00 INFO 11 --- [booklore-api] [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2025-11-30T14:32:46.003+01:00 INFO 11 --- [booklore-api] [nio-8080-exec-2] o.s.web.servlet.DispatcherServlet : Completed initialization in 1 ms
2025-11-30T14:33:03.148+01:00 INFO 11 --- [booklore-api] [ virtual-136] c.a.b.s.library.FileAsBookProcessor : Processing file: NIST CSF 1.1 vs 2.0 (PUBLIC DRAFT).pdf
Exception in thread "" java.lang.OutOfMemoryError: Java heap space
at java.desktop/java.awt.image.DataBufferInt.
docker said CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 9d1760b5c891 booklore 0.16% 6.754GiB / 30.75GiB 21.96% 1.73MB / 2.26MB 139MB / 1.16MB 109
Ah, I see it. Good news, this should be addressed shortly (probably next release). Both #1688 and #1680 are about this; basically, what happened is the JVM ran out of memory and crashed, and since there wasn't any recovery, it simply stopped.
Apologizes for inconvenience.