Stirling-Tools
[Bug]: The sessionLimit configuration is not effective
Installation Method
Docker
The Problem
I modified the "sessionLimit" in "setting.yaml", I also saw in the startup log that sessionLimit was changed.
But when I tested converting PDF to Word, it could only process one file at a time. If one file was large and got stuck, all subsequent tasks would be stuck waiting for the previous file to be processed.
Is this a bug? Because I have modified the sessionLimit configuration. Or does the free version only allow one file to be processed concurrently at a time?
setting.yaml
processExecutor:
sessionLimit: # Process executor instances limits
libreOfficeSessionLimit: 10
pdfToHtmlSessionLimit: 10
qpdfSessionLimit: 40
tesseractSessionLimit: 10
pythonOpenCvSessionLimit: 80
weasyPrintSessionLimit: 160
installAppSessionLimit: 10
calibreSessionLimit: 10
timeoutMinutes: # Process executor timeout in minutes
libreOfficetimeoutMinutes: 30
pdfToHtmltimeoutMinutes: 20
pythonOpenCvtimeoutMinutes: 30
weasyPrinttimeoutMinutes: 30
installApptimeoutMinutes: 60
calibretimeoutMinutes: 30
tesseractTimeoutMinutes: 30
startup log
spdf_nwHd-1 | Copying original files without overwriting existing files
spdf_nwHd-1 | cp: can't stat '/usr/share/tesseract-ocr/5/tessdata/*': No such file or directory
spdf_nwHd-1 | running with JAVA_TOOL_OPTIONS -XX:+UnlockExperimentalVMOptions -XX:MaxRAMPercentage=75 -XX:InitiatingHeapOccupancyPercent=20 -XX:+G1PeriodicGCInvokesConcurrent -XX:G1PeriodicGCInterval=10000 -XX:+UseStringDeduplication -XX:G1PeriodicGCSystemLoadThreshold=70
spdf_nwHd-1 | Running Stirling PDF with DOCKER_ENABLE_SECURITY=false and VERSION_TAG=0.46.0
spdf_nwHd-1 | Setting permissions and ownership for necessary directories...
spdf_nwHd-1 | Picked up JAVA_TOOL_OPTIONS: -XX:+UnlockExperimentalVMOptions -XX:MaxRAMPercentage=75 -XX:InitiatingHeapOccupancyPercent=20 -XX:+G1PeriodicGCInvokesConcurrent -XX:G1PeriodicGCInterval=10000 -XX:+UseStringDeduplication -XX:G1PeriodicGCSystemLoadThreshold=70
spdf_nwHd-1 | INFO:unoserver:Starting unoserver 3.2.
spdf_nwHd-1 | INFO:unoserver:Command: /usr/bin/soffice --headless --invisible --nocrashreport --nodefault --nologo --nofirststartwizard --norestore -env:UserInstallation=file:///tmp/tmp4lpnhkmf --accept=socket,host=127.0.0.1,port=2002,tcpNoDelay=1;urp;StarOffice.ComponentContext
spdf_nwHd-1 | 01:08:40.658 [main] INFO s.s.SPDF.config.ConfigInitializer - Settings file updated based on template changes.
spdf_nwHd-1 | 01:08:40.660 [main] INFO s.software.SPDF.SPDFApplication - Settings file: ./configs/settings.yml
spdf_nwHd-1 | 01:08:40.662 [main] INFO s.software.SPDF.SPDFApplication - Custom settings file: ./configs/custom_settings.yml
spdf_nwHd-1 | ____ _____ ___ ____ _ ___ _ _ ____ ____ ____ _____
spdf_nwHd-1 | / ___|_ _|_ _| _ \| | |_ _| \ | |/ ___| | _ \| _ \| ___|
spdf_nwHd-1 | \___ \ | | | || |_) | | | || \| | | _ _____| |_) | | | | |_
spdf_nwHd-1 | ___) || | | || _ <| |___ | || |\ | |_| |_____| __/| |_| | _|
spdf_nwHd-1 | |____/ |_| |___|_| \_\_____|___|_| \_|\____| |_| |____/|_|
spdf_nwHd-1 | Powered by Spring Boot 3.4.5
spdf_nwHd-1 | 01:08:41.489 [main] INFO s.software.SPDF.SPDFApplication - Starting SPDFApplication v0.46.0 using Java 21.0.7 with PID 21 (/app.jar started by stirlingpdfuser in /)
spdf_nwHd-1 | 01:08:41.491 [main] INFO s.software.SPDF.SPDFApplication - The following 1 profile is active: "default"
spdf_nwHd-1 | INFO:unoserver:Starting UnoConverter.
spdf_nwHd-1 | INFO:unoserver:Starting UnoComparer.
spdf_nwHd-1 | INFO:unoserver:Started.
spdf_nwHd-1 | 01:08:44.738 [main] INFO s.software.SPDF.SPDFApplication - Running configs ApplicationProperties(legal=ApplicationProperties.Legal(termsAndConditions=https://www.stirlingpdf.com/terms-and-conditions, privacyPolicy=https://www.stirlingpdf.com/privacy-policy, accessibilityStatement=, cookiePolicy=, impressum=), security=ApplicationProperties.Security(enableLogin=false, csrfDisabled=false, initialLogin=ApplicationProperties.Security.InitialLogin(username=), oauth2=ApplicationProperties.Security.OAUTH2(enabled=false, issuer=, clientId=, autoCreateUser=true, blockRegistration=false, useAsUsername=email, scopes=[openid, profile, email], provider=google, client=ApplicationProperties.Security.OAUTH2.Client(google=Google [clientId=, clientSecret=NULL, scopes=[email, profile], useAsUsername=EMAIL], github=GitHub [clientId=, clientSecret=NULL, scopes=[read:user], useAsUsername=LOGIN], keycloak=Keycloak [issuer=, clientId=, clientSecret=NULL, scopes=[openid, profile, email], useAsUsername=PREFERRED_USERNAME])), saml2=ApplicationProperties.Security.SAML2(provider=, enabled=false, autoCreateUser=true, blockRegistration=false, registrationId=stirling, idpSingleLogoutUrl=https://dev-XXXXXXXX.okta.com/app/dev-XXXXXXXX_stirlingpdf_1/externalKey/slo/saml, idpSingleLoginUrl=https://dev-XXXXXXXX.okta.com/app/dev-XXXXXXXX_stirlingpdf_1/externalKey/sso/saml, idpIssuer=, idpCert=class path resource [okta.cert]), loginAttemptCount=5, loginResetTimeMinutes=120, loginMethod=all, customGlobalAPIKey=null), system=ApplicationProperties.System(defaultLocale=en-US, googlevisibility=false, showUpdate=false, showUpdateOnlyAdmin=false, customHTMLFiles=false, tessdataDir=/usr/share/tessdata, enableAlphaFunctionality=false, enableAnalytics=true, datasource=ApplicationProperties.Datasource(enableCustomDatabase=false, customDatabaseUrl=, type=postgresql, hostName=localhost, port=5432, name=postgres, username=postgres), disableSanitize=false, enableUrlToPDF=false, customPaths=ApplicationProperties.CustomPaths(pipeline=ApplicationProperties.CustomPaths.Pipeline(watchedFoldersDir=, finishedFoldersDir=, webUIConfigsDir=null), operations=ApplicationProperties.CustomPaths.Operations(weasyprint=, unoconvert=)), fileUploadLimit=), ui=ApplicationProperties.Ui(appName=Fenger PDF, homeDescription=null, appNameNavbar=null, languages=[]), endpoints=ApplicationProperties.Endpoints(toRemove=[], groupsToRemove=[]), metrics=ApplicationProperties.Metrics(enabled=true), automaticallyGenerated=ApplicationProperties.AutomaticallyGenerated(UUID=f73a1f6c-659f-49f8-b3b0-0b42936785e2, appVersion=0.46.0), premium=ApplicationProperties.Premium(enabled=false, maxUsers=0, proFeatures=ApplicationProperties.Premium.ProFeatures(ssoAutoLogin=false, customMetadata=ApplicationProperties.Premium.ProFeatures.CustomMetadata(autoUpdateMetadata=false, author=username, creator=Stirling-PDF, producer=Stirling-PDF), googleDrive=ApplicationProperties.Premium.ProFeatures.GoogleDrive(enabled=false, clientId=, apiKey=, appId=)), enterpriseFeatures=ApplicationProperties.Premium.EnterpriseFeatures(persistentMetrics=ApplicationProperties.Premium.EnterpriseFeatures.PersistentMetrics(enabled=false, retentionDays=0))), enterpriseEdition=ApplicationProperties.EnterpriseEdition(enabled=false, maxUsers=0, ssoAutoLogin=false, customMetadata=ApplicationProperties.EnterpriseEdition.CustomMetadata(autoUpdateMetadata=false, author=null, creator=Stirling-PDF, producer=Stirling-PDF)), autoPipeline=ApplicationProperties.AutoPipeline(outputFolder=null), processExecutor=ApplicationProperties.ProcessExecutor(sessionLimit=ApplicationProperties.ProcessExecutor.SessionLimit(libreOfficeSessionLimit=10, pdfToHtmlSessionLimit=10, pythonOpenCvSessionLimit=80, weasyPrintSessionLimit=160, installAppSessionLimit=10, calibreSessionLimit=10, qpdfSessionLimit=40, tesseractSessionLimit=10), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30)))
spdf_nwHd-1 | 01:08:45.147 [main] INFO s.s.S.config.EndpointConfiguration - Total disabled endpoints: 1. Disabled endpoints: url-to-pdf
spdf_nwHd-1 | 01:08:45.631 [main] INFO s.software.SPDF.utils.FileMonitor - Monitoring directory: ./pipeline/watchedFolders
spdf_nwHd-1 | INFO:unoserver:Server PID: 44
spdf_nwHd-1 | 01:08:47.355 [main] INFO s.software.SPDF.SPDFApplication - Started SPDFApplication in 6.689 seconds (process running for 8.855)
spdf_nwHd-1 | 01:08:47.360 [scheduling-1] WARN s.software.SPDF.utils.FileMonitor - not monitoring any directory, even the root directory itself: ./pipeline/watchedFolders
spdf_nwHd-1 | 01:08:47.361 [scheduling-1] INFO s.software.SPDF.utils.FileMonitor - Registered directory: ./pipeline/watchedFolders
spdf_nwHd-1 | 01:08:47.366 [main] INFO s.software.SPDF.SPDFApplication - Stirling-PDF Started.
spdf_nwHd-1 | 01:08:47.367 [main] INFO s.software.SPDF.SPDFApplication - Navigate to http://localhost:8080/
We can see that sessionLimit was changed
(libreOfficeSessionLimit=10, pdfToHtmlSessionLimit=10, pythonOpenCvSessionLimit=80, weasyPrintSessionLimit=160, installAppSessionLimit=10, calibreSessionLimit=10, qpdfSessionLimit=40, tesseractSessionLimit=10), timeoutMinutes=ApplicationProperties.ProcessExecutor.TimeoutMinutes(libreOfficeTimeoutMinutes=30, pdfToHtmlTimeoutMinutes=20, pythonOpenCvTimeoutMinutes=30, weasyPrintTimeoutMinutes=30, installAppTimeoutMinutes=60, calibreTimeoutMinutes=30, tesseractTimeoutMinutes=30, qpdfTimeoutMinutes=30)))
Version of Stirling-PDF
0.46.0
Last Working Version of Stirling-PDF
No response
Page Where the Problem Occurred
No response
Docker Configuration
Relevant Log Output
Additional Information
No response
Browsers Affected
No response
No Duplicate of the Issue
- [x] I have verified that there are no existing issues raised related to my problem.
Hello @fengerwoo! I'm Dosu, a bot here to help you with bugs, answer your questions, and assist you in becoming a contributor. I'm ready to assist you while we wait for a human maintainer. How can I help you today?
The issue you're experiencing with the sessionLimit configuration in Stirling-PDF version 0.46.0 is not a bug or a limitation of the free version. The sessionLimit is applied per server deployment, meaning the limit is enforced across all users on the server, not per individual user. Additionally, the current web UI processes all files and operations in serial mode, which means that even with higher session limits, files are not processed concurrently. This is a limitation of the current web UI design [1].
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hello @fengerwoo! I'm Dosu, a bot here to help you with bugs, answer your questions, and assist you in becoming a contributor. I'm ready to assist you while we wait for a human maintainer. How can I help you today?
The issue you're experiencing with the
sessionLimitconfiguration in Stirling-PDF version 0.46.0 is not a bug or a limitation of the free version. ThesessionLimitis applied per server deployment, meaning the limit is enforced across all users on the server, not per individual user. Additionally, the current web UI processes all files and operations in serial mode, which means that even with higher session limits, files are not processed concurrently. This is a limitation of the current web UI design [1].To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
How can I process files in parallel? [@dosu]
Stirling-PDF currently processes all files and operations in serial mode, which means it does not support concurrent processing through the web UI. The sessionLimit configuration is applied per server deployment, not per user, which further restricts concurrent processing capabilities [1].
To achieve parallel processing, you might need to consider implementing a custom solution outside the web UI, such as using a script or external tool to manage multiple instances of Stirling-PDF, each handling different tasks concurrently. However, this would require additional setup and management.
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
Stirling-PDF currently processes all files and operations in serial mode, which means it does not support concurrent processing through the web UI. The
sessionLimitconfiguration is applied per server deployment, not per user, which further restricts concurrent processing capabilities [1].To achieve parallel processing, you might need to consider implementing a custom solution outside the web UI, such as using a script or external tool to manage multiple instances of Stirling-PDF, each handling different tasks concurrently. However, this would require additional setup and management.
To continue talking to Dosu, mention @dosu.
Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other
Do you mean, I can start multiple Stirling-PDF instances similar to: docker, and then use Nginx load balancing reverse proxy to use? If so, what is the meaning of sessionLimit configuration?
It is also worth mentioning that I want to use http API to convert files in parallel, not Web UI, and currently the api can only be serialized [@dosu]
Please ignore the bot 😂 They are pretending to know things, I will investigate your ticket and get back to you
