Stirling-PDF icon indicating copy to clipboard operation
Stirling-PDF copied to clipboard
verified publisher icon Stirling-Tools

[Bug]: The compression effect of compress-pdf is not good, and the memory it occupies is not released

Open Vincentauyeung opened this issue 1 year ago • 8 comments

Installation Method

Docker

The Problem

In version 0.36.4, the compression effect on files isn't significantly different. The same file had a compression ratio of 10% in version 0.33.1. Additionally, in version 0.36.4, the memory usage by Docker is relatively high and is not released after the task is completed, which can lead to failures when there are many tasks.

Version of Stirling-PDF

0.36.4

Last Working Version of Stirling-PDF

0.33.1

Page Where the Problem Occurred

No response

Docker Configuration

No response

Relevant Log Output

Load average: 0.01 0.01 0.00 2/731 555
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
   10     7 stirling S    21.7g  87%  10   0% java -Dfile.encoding=UTF-8 -jar /app.jar
  545     0 root     S     2620   0%  14   0% /bin/bash
    7     1 root     S     2320   0%   2   0% {init.sh} /bin/bash /scripts/init.sh java -Dfile.encoding=UTF-8 -jar /app.jar
  555   545 root     R     1628   0%   3   0% top
    1     0 root     S      848   0%   0   0% tini -- /scripts/init.sh java -Dfile.encoding=UTF-8 -jar /app.jar
fe0157f9c2ab:/#

08:06:43.306 [qtp1281580764-83] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_14486430207505780250.pdf /tmp/output_7797567749138520850.pdf
08:15:42.426 [qtp1281580764-119] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_6075460368366413101.pdf /tmp/output_2754503219077468122.pdf
08:25:39.981 [qtp1281580764-85] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_7063201442132836996.pdf /tmp/output_17154865287089274309.pdf
08:25:58.112 [qtp1281580764-85] WARN  s.s.S.c.api.misc.CompressController - Optimized file is larger than the original. Returning the original file instead.
08:27:39.299 [qtp1281580764-67] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_2216131900555302994.pdf, this will produce a corrupted file if you're also reading from it
08:27:39.878 [qtp1281580764-67] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=8 --compress-streams=y --object-streams=generate /tmp/input_2216131900555302994.pdf /tmp/output_6414094566086155025.pdf
08:27:39.993 [Thread-75] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_2216131900555302994.pdf: reported number of objects (4191) is not one plus the highest object number (4189)
08:27:57.431 [Thread-75] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
08:27:57.577 [qtp1281580764-67] WARN  s.s.S.c.api.misc.CompressController - Optimized file is larger than the original. Returning the original file instead.
08:52:43.399 [qtp1281580764-83] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_17373107469744773985.pdf, this will produce a corrupted file if you're also reading from it
08:52:43.779 [qtp1281580764-83] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=8 --compress-streams=y --object-streams=generate /tmp/input_17373107469744773985.pdf /tmp/output_1443993403229931841.pdf
08:52:43.797 [Thread-88] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_17373107469744773985.pdf: reported number of objects (4191) is not one plus the highest object number (4189)
08:53:00.789 [Thread-88] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
08:53:01.013 [qtp1281580764-83] WARN  s.s.S.c.api.misc.CompressController - Optimized file is larger than the original. Returning the original file instead.
Copying original files without overwriting existing files
Running Stirling PDF with DOCKER_ENABLE_SECURITY=false and VERSION_TAG=0.36.4
Setting permissions and ownership for necessary directories...
Picked up JAVA_TOOL_OPTIONS:  -XX:MaxRAMPercentage=75
 ____ _____ ___ ____  _     ___ _   _  ____       ____  ____  _____
/ ___|_   _|_ _|  _ \| |   |_ _| \ | |/ ___|     |  _ \|  _ \|  ___|
\___ \ | |  | || |_) | |    | ||  \| | |  _ _____| |_) | | | | |_
 ___) || |  | ||  _ <| |___ | || |\  | |_| |_____|  __/| |_| |  _|
|____/ |_| |___|_| \_\_____|___|_| \_|\____|     |_|   |____/|_|
Powered by Spring Boot 3.4.0
09:04:10.919 [main] INFO  s.software.SPDF.SPdfApplication - Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /)
09:04:10.923 [main] INFO  s.software.SPDF.SPdfApplication - The following 1 profile is active: "default"
09:04:17.398 [main] INFO  s.software.SPDF.SPdfApplication - Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@5fdb7394, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.36.4), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30)))
09:04:23.866 [main] INFO  s.s.S.config.EndpointConfiguration - Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline
Invalid APP_LOCALE environment variable value. Falling back to default Locale.UK.
09:04:28.990 [main] INFO  s.software.SPDF.SPdfApplication - Started SPdfApplication in 20.543 seconds (process running for 24.775)
09:04:29.014 [scheduling-1] WARN  s.software.SPDF.utils.FileMonitor - not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders
09:04:29.016 [scheduling-1] INFO  s.software.SPDF.utils.FileMonitor - Registered directory: ./pipeline/watchedFolders
09:04:29.020 [main] INFO  s.software.SPDF.SPdfApplication - Stirling-PDF Started.
09:04:29.020 [main] INFO  s.software.SPDF.SPdfApplication - Navigate to http://localhost:8080
09:17:02.012 [qtp1402479907-51] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_15453331872551591584.pdf /tmp/output_10424284958091912330.pdf
09:18:23.687 [qtp1402479907-75] WARN  o.e.jetty.io.AbstractConnection - Failed callback
java.io.IOException: Insufficient content written 1079345152 < 1153490315
        at org.eclipse.jetty.ee10.servlet.ServletChannel.sendErrorOrAbort(ServletChannel.java:615)
        at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:570)
        at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
        at org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717)
        at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060)
        at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740)
        at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81)
        at org.eclipse.jetty.server.Server.handle(Server.java:182)
        at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662)
        at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:418)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99)
        at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164)
        at java.base/java.lang.Thread.run(Thread.java:1583)
09:34:09.516 [qtp1402479907-84] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_11759335196431308915.pdf /tmp/output_2497789041052486310.pdf
09:38:04.025 [qtp1402479907-84] WARN  s.s.S.c.api.misc.CompressController - Optimized file is larger than the original. Returning the original file instead.
09:51:18.831 [qtp1402479907-72] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_9929772336105117985.pdf, this will produce a corrupted file if you're also reading from it
09:51:55.983 [qtp1402479907-72] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_9929772336105117985.pdf /tmp/output_3871698498268098119.pdf
09:51:56.155 [Thread-46] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_9929772336105117985.pdf: reported number of objects (158488) is not one plus the highest object number (158486)
09:59:43.868 [Thread-46] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
09:59:47.817 [qtp1402479907-72] WARN  s.s.S.c.api.misc.CompressController - Optimized file is larger than the original. Returning the original file instead.
Copying original files without overwriting existing files
Running Stirling PDF with DOCKER_ENABLE_SECURITY=false and VERSION_TAG=0.36.4
Setting permissions and ownership for necessary directories...
Picked up JAVA_TOOL_OPTIONS:  -XX:MaxRAMPercentage=75
 ____ _____ ___ ____  _     ___ _   _  ____       ____  ____  _____
/ ___|_   _|_ _|  _ \| |   |_ _| \ | |/ ___|     |  _ \|  _ \|  ___|
\___ \ | |  | || |_) | |    | ||  \| | |  _ _____| |_) | | | | |_
 ___) || |  | ||  _ <| |___ | || |\  | |_| |_____|  __/| |_| |  _|
|____/ |_| |___|_| \_\_____|___|_| \_|\____|     |_|   |____/|_|
Powered by Spring Boot 3.4.0
01:03:59.670 [main] INFO  s.software.SPDF.SPdfApplication - Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /)
01:03:59.705 [main] INFO  s.software.SPDF.SPdfApplication - The following 1 profile is active: "default"
01:04:05.640 [main] INFO  s.software.SPDF.SPdfApplication - Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@6248cfab, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.36.4), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30)))
01:04:11.803 [main] INFO  s.s.S.config.EndpointConfiguration - Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline
Invalid APP_LOCALE environment variable value. Falling back to default Locale.UK.
01:04:16.773 [main] INFO  s.software.SPDF.SPdfApplication - Started SPdfApplication in 19.413 seconds (process running for 22.637)
01:04:16.804 [scheduling-1] WARN  s.software.SPDF.utils.FileMonitor - not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders
01:04:16.806 [scheduling-1] INFO  s.software.SPDF.utils.FileMonitor - Registered directory: ./pipeline/watchedFolders
01:04:16.813 [main] INFO  s.software.SPDF.SPdfApplication - Stirling-PDF Started.
01:04:16.814 [main] INFO  s.software.SPDF.SPdfApplication - Navigate to http://localhost:8080
Copying original files without overwriting existing files
Running Stirling PDF with DOCKER_ENABLE_SECURITY=false and VERSION_TAG=0.36.4
Setting permissions and ownership for necessary directories...
Picked up JAVA_TOOL_OPTIONS:  -XX:MaxRAMPercentage=75
 ____ _____ ___ ____  _     ___ _   _  ____       ____  ____  _____
/ ___|_   _|_ _|  _ \| |   |_ _| \ | |/ ___|     |  _ \|  _ \|  ___|
\___ \ | |  | || |_) | |    | ||  \| | |  _ _____| |_) | | | | |_
 ___) || |  | ||  _ <| |___ | || |\  | |_| |_____|  __/| |_| |  _|
|____/ |_| |___|_| \_\_____|___|_| \_|\____|     |_|   |____/|_|
Powered by Spring Boot 3.4.0
01:13:27.117 [main] INFO  s.software.SPDF.SPdfApplication - Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /)
01:13:27.122 [main] INFO  s.software.SPDF.SPdfApplication - The following 1 profile is active: "default"
01:13:32.105 [main] INFO  s.software.SPDF.SPdfApplication - Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@5fdb7394, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.36.4), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30)))
01:13:32.679 [main] INFO  s.s.S.config.EndpointConfiguration - Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline
Invalid APP_LOCALE environment variable value. Falling back to default Locale.UK.
01:13:36.460 [main] INFO  s.software.SPDF.SPdfApplication - Started SPdfApplication in 10.903 seconds (process running for 12.847)
01:13:36.467 [scheduling-1] WARN  s.software.SPDF.utils.FileMonitor - not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders
01:13:36.471 [scheduling-1] INFO  s.software.SPDF.utils.FileMonitor - Registered directory: ./pipeline/watchedFolders
01:13:36.481 [main] INFO  s.software.SPDF.SPdfApplication - Stirling-PDF Started.
01:13:36.482 [main] INFO  s.software.SPDF.SPdfApplication - Navigate to http://localhost:8080
02:18:19.248 [qtp1402479907-49] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_1709220522492119263.pdf, this will produce a corrupted file if you're also reading from it
02:18:30.831 [qtp1402479907-49] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=8 --compress-streams=y --object-streams=generate /tmp/input_1709220522492119263.pdf /tmp/output_8417684886762377398.pdf
02:18:30.919 [Thread-29] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_1709220522492119263.pdf: reported number of objects (4826) is not one plus the highest object number (4824)
02:18:46.578 [Thread-29] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
03:12:35.565 [qtp1402479907-83] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_18322339359892560310.pdf, this will produce a corrupted file if you're also reading from it
03:12:42.620 [qtp1402479907-83] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_18322339359892560310.pdf /tmp/output_475909062257444646.pdf
03:12:43.179 [Thread-35] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_18322339359892560310.pdf: reported number of objects (4826) is not one plus the highest object number (4824)
03:12:52.797 [Thread-35] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
03:12:53.032 [qtp1402479907-83] INFO  s.s.S.c.api.misc.CompressController - Current compression ratio: 5.29
03:25:00.433 [qtp1402479907-83] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_18322339359892560310.pdf, this will produce a corrupted file if you're also reading from it
03:25:02.694 [qtp1402479907-83] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_18322339359892560310.pdf /tmp/output_475909062257444646.pdf
03:25:02.965 [Thread-37] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_18322339359892560310.pdf: reported number of objects (5637) is not one plus the highest object number (5635)
03:25:06.790 [Thread-37] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
03:25:06.834 [qtp1402479907-83] INFO  s.s.S.c.api.misc.CompressController - Current compression ratio: 1.40
03:28:21.709 [qtp1402479907-83] WARN  o.apache.pdfbox.pdmodel.PDDocument - You are overwriting the existing file input_18322339359892560310.pdf, this will produce a corrupted file if you're also reading from it
03:28:22.324 [qtp1402479907-83] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_18322339359892560310.pdf /tmp/output_475909062257444646.pdf
03:28:22.371 [Thread-39] INFO  s.s.SPDF.utils.ProcessExecutor - WARNING: /tmp/input_18322339359892560310.pdf: reported number of objects (6448) is not one plus the highest object number (6446)
03:28:23.985 [Thread-39] INFO  s.s.SPDF.utils.ProcessExecutor - qpdf: operation succeeded with warnings; resulting file may have some problems
03:50:36.465 [qtp1402479907-54] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=2 --compress-streams=y --object-streams=generate /tmp/input_3818756845783608715.pdf /tmp/output_11630587244097675436.pdf
03:56:58.371 [qtp1402479907-66] INFO  s.s.SPDF.utils.ProcessExecutor - Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=2 --compress-streams=y --object-streams=generate /tmp/input_9519703529233364227.pdf /tmp/output_11004865621880217967.pdf
03:58:35.268 [qtp1402479907-70] WARN  o.e.jetty.io.AbstractConnection - Failed callback
java.io.IOException: Insufficient content written 1079803904 < 1153490315
        at org.eclipse.jetty.ee10.servlet.ServletChannel.sendErrorOrAbort(ServletChannel.java:615)
        at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:570)
        at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575)
        at org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717)
        at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060)
        at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740)
        at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81)
        at org.eclipse.jetty.server.Server.handle(Server.java:182)
        at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662)
        at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:418)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99)
        at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
        at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164)
        at java.base/java.lang.Thread.run(Thread.java:1583)

Additional Information

No response

Browsers Affected

No response

No Duplicate of the Issue

  • [X] I have verified that there are no existing issues raised related to my problem.

Vincentauyeung avatar Dec 20 '24 06:12 Vincentauyeung

@Vincentauyeung Hello there! I'm here to help you with any issues, answer questions, and guide you on how to become a contributor. If you need assistance, feel free to ask!

There is a similar open issue regarding high RAM usage in Stirling-PDF. The suggested solution involves configuring memory limits in Docker. For instance, setting a memory limit of 1GB in the docker-compose configuration can help manage RAM usage. Additionally, increasing the memory limit from 2GB to 4GB was suggested as a solution in a related issue [1].

Regarding the compression process, the qpdf command is used to optimize PDF files by applying various compression techniques, such as normalizing content, optimizing images, and recompressing Flate streams. The command also adjusts the compression level dynamically based on the desired output size [2]. If the optimized file is larger than the original, Stirling-PDF returns the original file instead [2].

For the memory not being released after tasks, you might want to ensure that temporary files are properly cleaned up. Stirling-PDF handles temporary files by deleting them after the process completes, which should help in managing memory usage [3][4].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

dosubot[bot] avatar Dec 20 '24 06:12 dosubot[bot]

I see you inputted multiple pdfs and that some errored Are you able to share any pdfs that had this error or what caused it in your logs?

Additionally did all pdfs compress badly that you tried , even compression level #10 etc

Frooodle avatar Dec 20 '24 08:12 Frooodle

I tried several files using level 9 compression, but the results were not good. However, these files could be compressed to 10% using level 3 in version 0.33.1.

2024-12-19 00:00:00,007 INFO s.s.S.c.s.d.DatabaseBackupHelper [scheduling-1] Deleted oldest backup: backup_202412150000.sql 2024-12-19 00:00:00,083 INFO s.s.S.c.s.d.DatabaseBackupHelper [scheduling-1] Database export completed: configs/db/backup/backup_202412190000.sql 2024-12-19 06:52:16,930 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.33.1 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-19 06:52:16,936 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-19 06:52:21,094 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.36.0 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-19 06:52:21,099 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-19 06:52:22,167 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@247bbfba, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, ocrMyPdfSessionLimit=2, pythonOpenCvSessionLimit=8, ghostScriptSessionLimit=16, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, ocrMyPdfTimeoutMinutes=30, pythonOpenCvTimeoutMinutes=30, ghostScriptTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30))) 2024-12-19 06:52:23,375 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 3. Disabled endpoints: book-to-pdf, pdf-to-book, pipeline 2024-12-19 06:52:25,921 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 10.571 seconds (process running for 12.464) 2024-12-19 06:52:25,973 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-19 06:52:25,975 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-19 06:52:25,981 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-19 06:52:25,985 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-19 06:52:27,325 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@6aa6c17, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30))) 2024-12-19 06:52:33,269 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 3. Disabled endpoints: book-to-pdf, pdf-to-book, pipeline 2024-12-19 06:52:37,492 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 18.335 seconds (process running for 22.041) 2024-12-19 06:52:37,507 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-19 06:52:37,510 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-19 06:52:37,517 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-19 06:52:37,518 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-19 07:24:33,240 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.33.1 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-19 07:24:33,246 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-19 07:24:35,982 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-19 07:24:35,989 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-19 07:24:38,300 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@3e28dc96, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, ocrMyPdfSessionLimit=2, pythonOpenCvSessionLimit=8, ghostScriptSessionLimit=16, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, ocrMyPdfTimeoutMinutes=30, pythonOpenCvTimeoutMinutes=30, ghostScriptTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30))) 2024-12-19 07:24:39,543 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 3. Disabled endpoints: book-to-pdf, pdf-to-book, pipeline 2024-12-19 07:24:42,028 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@573a078, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.35.0), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30))) 2024-12-19 07:24:42,563 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 10.861 seconds (process running for 12.803) 2024-12-19 07:24:42,586 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-19 07:24:42,589 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-19 07:24:42,589 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-19 07:24:42,590 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-19 07:24:42,831 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline 2024-12-19 07:24:46,465 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 12.018 seconds (process running for 14.366) 2024-12-19 07:24:46,482 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-19 07:24:46,486 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-19 07:24:46,490 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-19 07:24:46,491 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-19 07:35:17,216 INFO s.s.S.u.ProcessExecutor [qtp1281580764-52] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=4 --compress-streams=y --object-streams=generate /tmp/input_9101584447575576667.pdf /tmp/output_13473992372699728362.pdf 2024-12-19 07:43:20,501 INFO s.s.S.u.ProcessExecutor [qtp1281580764-54] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_624923748996283878.pdf /tmp/output_6292785540531933282.pdf 2024-12-19 07:43:20,526 INFO s.s.S.u.ProcessExecutor [Thread-30] WARNING: /tmp/input_624923748996283878.pdf: reported number of objects (233) is not one plus the highest object number (231) 2024-12-19 07:43:25,116 INFO s.s.S.u.ProcessExecutor [Thread-30] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-19 07:43:25,138 WARN s.s.S.c.a.m.CompressController [qtp1281580764-54] Optimized file is larger than the original. Returning the original file instead. 2024-12-19 07:47:10,232 INFO s.s.S.u.ProcessExecutor [qtp1281580764-49] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=4 --compress-streams=y --object-streams=generate /tmp/input_8368114918855889468.pdf /tmp/output_2393118544895736053.pdf 2024-12-19 07:48:35,443 WARN o.e.j.i.AbstractConnection [qtp1281580764-84] Failed callback java.io.IOException: Insufficient content written 1079083008 < 1153490315 at org.eclipse.jetty.ee10.servlet.ServletChannel.sendErrorOrAbort(ServletChannel.java:615) at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:570) at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575) at org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060) at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740) at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:418) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164) at java.base/java.lang.Thread.run(Thread.java:1583) 2024-12-19 07:54:04,954 INFO s.s.S.u.ProcessExecutor [qtp1281580764-68] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=4 --compress-streams=y --object-streams=generate /tmp/input_6234246292449022770.pdf /tmp/output_14516297432345590262.pdf 2024-12-19 07:55:36,491 WARN o.e.j.i.AbstractConnection [qtp1281580764-103] Failed callback java.io.IOException: Insufficient content written 1078099968 < 1153490315 at org.eclipse.jetty.ee10.servlet.ServletChannel.sendErrorOrAbort(ServletChannel.java:615) at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:570) at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575) at org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060) at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740) at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:418) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164) at java.base/java.lang.Thread.run(Thread.java:1583) 2024-12-19 08:06:43,306 INFO s.s.S.u.ProcessExecutor [qtp1281580764-83] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_14486430207505780250.pdf /tmp/output_7797567749138520850.pdf 2024-12-19 08:15:42,426 INFO s.s.S.u.ProcessExecutor [qtp1281580764-119] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_6075460368366413101.pdf /tmp/output_2754503219077468122.pdf 2024-12-19 08:25:39,981 INFO s.s.S.u.ProcessExecutor [qtp1281580764-85] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_7063201442132836996.pdf /tmp/output_17154865287089274309.pdf 2024-12-19 08:25:58,112 WARN s.s.S.c.a.m.CompressController [qtp1281580764-85] Optimized file is larger than the original. Returning the original file instead. 2024-12-19 08:27:39,299 WARN o.a.p.p.PDDocument [qtp1281580764-67] You are overwriting the existing file input_2216131900555302994.pdf, this will produce a corrupted file if you're also reading from it 2024-12-19 08:27:39,878 INFO s.s.S.u.ProcessExecutor [qtp1281580764-67] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=8 --compress-streams=y --object-streams=generate /tmp/input_2216131900555302994.pdf /tmp/output_6414094566086155025.pdf 2024-12-19 08:27:39,993 INFO s.s.S.u.ProcessExecutor [Thread-75] WARNING: /tmp/input_2216131900555302994.pdf: reported number of objects (4191) is not one plus the highest object number (4189) 2024-12-19 08:27:57,431 INFO s.s.S.u.ProcessExecutor [Thread-75] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-19 08:27:57,577 WARN s.s.S.c.a.m.CompressController [qtp1281580764-67] Optimized file is larger than the original. Returning the original file instead. 2024-12-19 08:52:43,399 WARN o.a.p.p.PDDocument [qtp1281580764-83] You are overwriting the existing file input_17373107469744773985.pdf, this will produce a corrupted file if you're also reading from it 2024-12-19 08:52:43,779 INFO s.s.S.u.ProcessExecutor [qtp1281580764-83] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=8 --compress-streams=y --object-streams=generate /tmp/input_17373107469744773985.pdf /tmp/output_1443993403229931841.pdf 2024-12-19 08:52:43,797 INFO s.s.S.u.ProcessExecutor [Thread-88] WARNING: /tmp/input_17373107469744773985.pdf: reported number of objects (4191) is not one plus the highest object number (4189) 2024-12-19 08:53:00,789 INFO s.s.S.u.ProcessExecutor [Thread-88] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-19 08:53:01,013 WARN s.s.S.c.a.m.CompressController [qtp1281580764-83] Optimized file is larger than the original. Returning the original file instead. 2024-12-19 09:04:10,919 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-19 09:04:10,923 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-19 09:04:17,398 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@5fdb7394, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.36.4), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30))) 2024-12-19 09:04:23,866 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline 2024-12-19 09:04:28,990 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 20.543 seconds (process running for 24.775) 2024-12-19 09:04:29,014 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-19 09:04:29,016 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-19 09:04:29,020 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-19 09:04:29,020 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-19 09:17:02,012 INFO s.s.S.u.ProcessExecutor [qtp1402479907-51] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_15453331872551591584.pdf /tmp/output_10424284958091912330.pdf 2024-12-19 09:18:23,687 WARN o.e.j.i.AbstractConnection [qtp1402479907-75] Failed callback java.io.IOException: Insufficient content written 1079345152 < 1153490315 at org.eclipse.jetty.ee10.servlet.ServletChannel.sendErrorOrAbort(ServletChannel.java:615) at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:570) at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575) at org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060) at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740) at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:418) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164) at java.base/java.lang.Thread.run(Thread.java:1583) 2024-12-19 09:34:09,516 INFO s.s.S.u.ProcessExecutor [qtp1402479907-84] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=5 --compress-streams=y --object-streams=generate /tmp/input_11759335196431308915.pdf /tmp/output_2497789041052486310.pdf 2024-12-19 09:38:04,025 WARN s.s.S.c.a.m.CompressController [qtp1402479907-84] Optimized file is larger than the original. Returning the original file instead. 2024-12-19 09:51:18,831 WARN o.a.p.p.PDDocument [qtp1402479907-72] You are overwriting the existing file input_9929772336105117985.pdf, this will produce a corrupted file if you're also reading from it 2024-12-19 09:51:55,983 INFO s.s.S.u.ProcessExecutor [qtp1402479907-72] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_9929772336105117985.pdf /tmp/output_3871698498268098119.pdf 2024-12-19 09:51:56,155 INFO s.s.S.u.ProcessExecutor [Thread-46] WARNING: /tmp/input_9929772336105117985.pdf: reported number of objects (158488) is not one plus the highest object number (158486) 2024-12-19 09:59:43,868 INFO s.s.S.u.ProcessExecutor [Thread-46] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-19 09:59:47,817 WARN s.s.S.c.a.m.CompressController [qtp1402479907-72] Optimized file is larger than the original. Returning the original file instead.

2024-12-20 01:03:59,670 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-20 01:03:59,705 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-20 01:04:05,640 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@6248cfab, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.36.4), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30))) 2024-12-20 01:04:11,803 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline 2024-12-20 01:04:16,773 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 19.413 seconds (process running for 22.637) 2024-12-20 01:04:16,804 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-20 01:04:16,806 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-20 01:04:16,813 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-20 01:04:16,814 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-20 01:13:27,117 INFO s.s.S.SPdfApplication [main] Starting SPdfApplication v0.36.4 using Java 21.0.5 with PID 10 (/app.jar started by stirlingpdfuser in /) 2024-12-20 01:13:27,122 INFO s.s.S.SPdfApplication [main] The following 1 profile is active: "default" 2024-12-20 01:13:32,105 INFO s.s.S.SPdfApplication [main] Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=true, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=false, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[https://www.googleapis.com/auth/userinfo.email, https://www.googleapis.com/auth/userinfo.profile], useAsUsername=email], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=login], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=preferred_username])), saml2=stirling.software.SPDF.model.ApplicationProperties$Security$SAML2@5fdb7394, loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=zh_CN, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=false), ui=ApplicationProperties.Ui(appName=H-PDF, homeDescription=null, appNameNavbar=null), endpoints=ApplicationProperties.Endpoints(toRemove=[pipeline], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=831afeab-f32b-4994-87ba-93e0aa920a0f, appVersion=0.36.4), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=1, pdfToHtmlSessionLimit=1, pythonOpenCvSessionLimit=8, weasyPrintSessionLimit=16, installAppSessionLimit=1, calibreSessionLimit=1, qpdfSessionLimit=4, tesseractSessionLimit=1), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30))) 2024-12-20 01:13:32,679 INFO s.s.S.c.EndpointConfiguration [main] Total disabled endpoints: 4. Disabled endpoints: book-to-pdf, pdf-to-book, pdf-to-pdfa, pipeline 2024-12-20 01:13:36,460 INFO s.s.S.SPdfApplication [main] Started SPdfApplication in 10.903 seconds (process running for 12.847) 2024-12-20 01:13:36,467 WARN s.s.S.u.FileMonitor [scheduling-1] not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders 2024-12-20 01:13:36,471 INFO s.s.S.u.FileMonitor [scheduling-1] Registered directory: ./pipeline/watchedFolders 2024-12-20 01:13:36,481 INFO s.s.S.SPdfApplication [main] Stirling-PDF Started. 2024-12-20 01:13:36,482 INFO s.s.S.SPdfApplication [main] Navigate to http://localhost:8080 2024-12-20 02:18:19,248 WARN o.a.p.p.PDDocument [qtp1402479907-49] You are overwriting the existing file input_1709220522492119263.pdf, this will produce a corrupted file if you're also reading from it 2024-12-20 02:18:30,831 INFO s.s.S.u.ProcessExecutor [qtp1402479907-49] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=8 --compress-streams=y --object-streams=generate /tmp/input_1709220522492119263.pdf /tmp/output_8417684886762377398.pdf 2024-12-20 02:18:30,919 INFO s.s.S.u.ProcessExecutor [Thread-29] WARNING: /tmp/input_1709220522492119263.pdf: reported number of objects (4826) is not one plus the highest object number (4824) 2024-12-20 02:18:46,578 INFO s.s.S.u.ProcessExecutor [Thread-29] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-20 03:12:35,565 WARN o.a.p.p.PDDocument [qtp1402479907-83] You are overwriting the existing file input_18322339359892560310.pdf, this will produce a corrupted file if you're also reading from it 2024-12-20 03:12:42,620 INFO s.s.S.u.ProcessExecutor [qtp1402479907-83] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_18322339359892560310.pdf /tmp/output_475909062257444646.pdf 2024-12-20 03:12:43,179 INFO s.s.S.u.ProcessExecutor [Thread-35] WARNING: /tmp/input_18322339359892560310.pdf: reported number of objects (4826) is not one plus the highest object number (4824) 2024-12-20 03:12:52,797 INFO s.s.S.u.ProcessExecutor [Thread-35] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-20 03:12:53,032 INFO s.s.S.c.a.m.CompressController [qtp1402479907-83] Current compression ratio: 5.29 2024-12-20 03:25:00,433 WARN o.a.p.p.PDDocument [qtp1402479907-83] You are overwriting the existing file input_18322339359892560310.pdf, this will produce a corrupted file if you're also reading from it 2024-12-20 03:25:02,694 INFO s.s.S.u.ProcessExecutor [qtp1402479907-83] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_18322339359892560310.pdf /tmp/output_475909062257444646.pdf 2024-12-20 03:25:02,965 INFO s.s.S.u.ProcessExecutor [Thread-37] WARNING: /tmp/input_18322339359892560310.pdf: reported number of objects (5637) is not one plus the highest object number (5635) 2024-12-20 03:25:06,790 INFO s.s.S.u.ProcessExecutor [Thread-37] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-20 03:25:06,834 INFO s.s.S.c.a.m.CompressController [qtp1402479907-83] Current compression ratio: 1.40 2024-12-20 03:28:21,709 WARN o.a.p.p.PDDocument [qtp1402479907-83] You are overwriting the existing file input_18322339359892560310.pdf, this will produce a corrupted file if you're also reading from it 2024-12-20 03:28:22,324 INFO s.s.S.u.ProcessExecutor [qtp1402479907-83] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=9 --compress-streams=y --object-streams=generate /tmp/input_18322339359892560310.pdf /tmp/output_475909062257444646.pdf 2024-12-20 03:28:22,371 INFO s.s.S.u.ProcessExecutor [Thread-39] WARNING: /tmp/input_18322339359892560310.pdf: reported number of objects (6448) is not one plus the highest object number (6446) 2024-12-20 03:28:23,985 INFO s.s.S.u.ProcessExecutor [Thread-39] qpdf: operation succeeded with warnings; resulting file may have some problems 2024-12-20 03:50:36,465 INFO s.s.S.u.ProcessExecutor [qtp1402479907-54] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=2 --compress-streams=y --object-streams=generate /tmp/input_3818756845783608715.pdf /tmp/output_11630587244097675436.pdf 2024-12-20 03:56:58,371 INFO s.s.S.u.ProcessExecutor [qtp1402479907-66] Running command: qpdf --normalize-content=y --optimize-images --recompress-flate --compression-level=2 --compress-streams=y --object-streams=generate /tmp/input_9519703529233364227.pdf /tmp/output_11004865621880217967.pdf 2024-12-20 03:58:35,268 WARN o.e.j.i.AbstractConnection [qtp1402479907-70] Failed callback java.io.IOException: Insufficient content written 1079803904 < 1153490315 at org.eclipse.jetty.ee10.servlet.ServletChannel.sendErrorOrAbort(ServletChannel.java:615) at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:570) at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:575) at org.eclipse.jetty.ee10.servlet.SessionHandler.handle(SessionHandler.java:717) at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060) at org.eclipse.jetty.server.Handler$Wrapper.handle(Handler.java:740) at org.eclipse.jetty.server.handler.EventsHandler.handle(EventsHandler.java:81) at org.eclipse.jetty.server.Server.handle(Server.java:182) at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662) at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:418) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164) at java.base/java.lang.Thread.run(Thread.java:1583)

Vincentauyeung avatar Dec 23 '24 01:12 Vincentauyeung

I attempted to compress the same 1GB file repeatedly, and after a few times, an error occurred. The likely reason is that memory is not released immediately after a task is completed, and multiple compressions lead to full memory usage。 Compressing a 1GB file requires 6GB of memory on the server. My server is configured with 24GB of memory, and after running 4 tasks, failures occur.

Vincentauyeung avatar Dec 23 '24 01:12 Vincentauyeung

any update?

Vincentauyeung avatar Jan 07 '25 04:01 Vincentauyeung

Same here, after trying to compress a few 2.6 MB files, my memory usage remains at 2.9GB and appears not to be released. The files used were digitally signed pdfs.

Toastyyy3 avatar Feb 27 '25 13:02 Toastyyy3

Investigating Also please note digitally signed pdf's can't be compressed as it will invalidate the signature

Frooodle avatar Feb 27 '25 15:02 Frooodle

Same issue, just compressed a 38 MB file at compression level 7 and RAM remains at 4.42 GB. Let me know if there is any way I can help troubleshoot

TechToet avatar May 09 '25 11:05 TechToet