No PDF generated, no log

[manu-p]

Hi,

I set up a PDF conversion flow as follows: When File created, File updated and File MIME type matches Office documents => Keep original, overwrite existing PDF

I'd expect that when creating a new odt file with Collabora Online Development Edition, or just uploading a new odt file, I would see its corresponding pdf appearing aside, but nothing happens.

And absolutely nothing in the nextcloug log (loglevel : 2).

libreoffice --headless --invisible --norestore --convert-to pdf file.odt works fine in a terminal.

PDF Converter app

PDF Converter app version: 1.3.1

Server configuration

Operating system: Ubuntu 18.04 Web server: nginx 1.14.0 Database: MariaDB 10.1.44 PHP version: PHP 7.4.6 Nextcloud Version: 18.0.4

LibreOffice 6.0.7

# which libreoffice
/usr/bin/libreoffice

List of activated apps: Enabled:

• accessibility: 1.4.0
• activity: 2.11.0
• admin_audit: 1.8.0
• apporder: 0.9.0
• bookmarks: 3.0.13
• breezedark: 18.0.16
• bruteforcesettings: 1.6.0
• calendar: 2.0.3
• cloud_federation_api: 1.1.0
• comments: 1.8.0
• contacts: 3.3.0
• dav: 1.14.0
• deck: 1.0.1
• external: 3.5.0
• federatedfilesharing: 1.8.0
• federation: 1.8.0
• files: 1.13.1
• files_accesscontrol: 1.8.1
• files_external: 1.9.0
• files_linkeditor: 1.0.13
• files_lock: 0.8.1
• files_mindmap: 0.0.21
• files_pdfviewer: 1.7.0
• files_rightclick: 0.15.2
• files_sharing: 1.10.1
• files_trashbin: 1.8.0
• files_versions: 1.11.0
• files_videoplayer: 1.7.0
• firstrunwizard: 2.7.0
• forms: 1.1.1
• groupfolders: 6.0.6
• impersonate: 1.5.0
• logreader: 2.3.0
• lookup_server_connector: 1.6.0
• nextcloud_announcements: 1.7.0
• notifications: 2.6.0
• oauth2: 1.6.0
• ownpad: 0.6.14
• password_policy: 1.8.0
• passwords: 2020.5.0
• photos: 1.0.0
• polls: 1.4.3
• previewgenerator: 2.3.0
• privacy: 1.2.0
• provisioning_api: 1.8.0
• quickaccesssorting: 1.0.3
• quota_warning: 1.7.0
• recommendations: 0.6.0
• richdocuments: 3.6.0
• serverinfo: 1.8.0
• settings: 1.0.0
• sharebymail: 1.8.0
• spreed: 8.0.9
• support: 1.1.0
• survey_client: 1.6.0
• systemtags: 1.8.0
• tasks: 0.13.1
• theming: 1.9.0
• theming_customcss: 1.5.0
• twofactor_backupcodes: 1.7.0
• updatenotification: 1.8.0
• viewer: 1.2.0
• workflow_pdf_converter: 1.3.1
• workflowengine: 2.0.0

Disabled:

• encryption
• end_to_end_encryption
• text
• user_ldap

Nextcloud configuration:

'enable_previews' => true, 'preview_libreoffice_path' => '/usr/bin/libreoffice', 'preview_office_cl_parameters' => ' --headless --invisible --norestore --convert-to pdf ',

Are you using external storage, if yes which one: local/smb/sftp/... No

Are you using encryption: yes/no No

Server log (data/nextcloud.log)

Nothing

[manu-p]

Any help appreciated. What more information can I provide to help solve this issue?

[manu-p]

Is there anybody out there?

[janikvonrotz]

I have similar setup and got the same problem. I am not sure about the cron.php. Does flow work at all in your case?

[janikvonrotz]

[manu-p]

@janikvonrotz yes, flow's working fine for "Block access to a file". We get no other flow setup at the moment.

[manu-p]

Whatever the background jobs method, no PDF is generated with this workflow.

Surprising there's no reaction here, apart yours, @janikvonrotz to confirm the problem.

Is this feature still supported?

[Schmuuu]

Well I gave a thumbs up, to support this issue and to show that I'm also affected :) I tried automated PDF creation via flow when NC 18 came out, but I never succeeded. I gave up at some point. I believed it was maybe my mistake and didn't file an issue.

[glorenzutti]

Im on the same page here. This dosent seems to work.

Also, I did a post on the help forum of nextcloud.

https://help.nextcloud.com/t/how-to-debug-a-workflow/84975

[manu-p]

Alright, thanks @glorenzutti Hope you get more luck than us before. :-)

[vinid223]

For some updates, I did comment on @glorenzutti post in the help forums to be able to figure out the problem.

Follow the forum thread in case the fix is found, but I don't really have high hopes.

[manu-p]

I already do that, following this help forum thread... ;-)

[rcdncn]

@manu-p back in March I tried to implement this and could not get any action in the log (or PDF in the folder). Following.

[vinid223]

@rcdncn You need to be patient, I found that the cronjob for conversion is executed each 5 minutes not as soon as a file is edited.

I did found that I get logs each time it tries to run, but the reasons array is empty since the shell_exec command just doesn't return anything. There might be a bug in the php version used, or just the command. I am not that familiar with PHP though.

[vinid223]

For the devs, here is what I did for testing and searching. It is from the post I commented on the Nextcloud Help forum.

I confirmed first that my entry for the Flow was well configured with someone from the community.

Since I was using Nextcloud in a docker, I had to install the libreoffice binary into the docker image to have it. I can use the “libreoffice” command anywhere in the image since it is configured to run in the path.

I changed a docx file a lot of time to test each changes I could have done. Even restart the docker container for nextcloud.

I’ve configures a flow to send messages to Talk to be sure that the flow cron updates did actually worked to be sure I didn’t messed with this feature. It did sent a message into a conversation each time I did something that fits the creteria of the PDF converter, which I copied into my other flow for talk message.

I was also curisous to find if it was something else I have done with my installation so I’ve looked at the source code for the extension to convert to PDF and here is what I found.

The extension code should be able to detect my libreoffice installation in this method https://github.com/nextcloud/workflow_pdf_converter/blob/6951eb123e0618b125c2aba1f15791345ae2d41e/lib/BackgroundJobs/Convert.php#L130-L147

The first step check if the configuration 'preview_libreoffice_path' => '/usr/bin/libreoffice' is documented which I have, but by default it is not. So normally, the next step check for the binary location with the command shell_exec('command -v libreoffice'); on line 136 of the Convert.php file in the BackgroundJobs folder of the lib directory.

I wanted to go further to do some more testing to prove the thing is not working. Here is what I did.

Setup:

Virtual Machine with

• Ubuntu server 20.04
• Installed Nextcloud 19.0 with snap
• Installed Libreoffice (tested the command command -v libreoffice to check the path returned and it did return the right path
• Installed the PDF Conversion APPS
• Did the following configuration image

• Installed the collabora core and apps to help me edit a docx file

When I update a file, the talk message is well received

But when I check to see if there is a PDF generated, nothing is happening. I looked at the logs and this is what is returned.

When I setup the config in the config.php, the error show "Could not convert file Test.docx, reason: []". There is no error in the reason array. Nothing is happening.

After looking again at the source code, it might just be the exec that always return a non 0 return code and never continues the save process. This might be the reason the array of reason is empty. I could debug the code, but I don’t really want to go that deep since I can reproduce the error with a fresh install of Nexcloud on docker or in a VM.

So I don’t know what the issue is really. More debug should be done, but at this point, I think the maintainers could check it out. The problem seems to be on any platform. Even the leaders on the forum told me that their installation stop generating PDFs for a while.

Hope this helps

[xDeerStalker]

Think i solved it. I added a logline in Converter.php to print $exec and then manually run it as www-data user. The issue is not in the scope of this app nor in the scope of nextcloud.

The underyling problem is that libreoffice will be run with www-data user and it needs the directory ~/.cache/dconf/

Since the home-directory of www-data usually is /var/www which is owned by root, www-data can´t create the directory and libreoffice fails to convert.

So either:

sudo mkdir /var/www/.cache/dconf/ sudo chown -R www-data:www-data /var/www/.cache sudo chown www-data:www-data /var/www

OR

change the home directory of www-data via passwd to somewhere else and create .cache/dconf/ there.

The first method probably has some security impacts whilest the second method probably has some stability impacts for software that relies on www-data home beeing /var/www

Maybe someone else knows a better solution? At least, both solutions provided should get the App working again

[manu-p]

Hi, glad to read it works for you (at least). But not for me (1st method).

Still no pdf generated (15 min cron job), still nothing in log files (nextcloud and nginx).

Maybe related, or not, I sometimes get a error mail from Cron: Subject: Cron <www-data@server> /usr/bin/php -f /var/www/nextcloud/cron.php Content: func=xmlSecCheckVersionExt:file=xmlsec.c:line=188:obj=unknown:subj=unknown:error=19:invalid version:mode=abi compatible;expected minor version=2;real minor version=2;expected subminor version=25;real subminor version=26

Seems to be related to the preview generator, but in case...

[xDeerStalker]

Okay, try the following:

1. upload a docx file named ConverterTest.docx to nextcloud
2. sudo -u www-data cp /var/www/nextcloud/data/path/to/ConverterTest.docx /tmp
3. sudo -u www-data /usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/ConverterTest.docx'

"3" is the exact command with the same priviledges that would be used by this app. Now there are 2 possible outcomes:

After the command has finished, /tmp/ConverterTest.pdf exists. If this is the case: sudo -u www-data cp /tmp/ConverterTest.pdf /var/www/nextcloud/data/path/to/ and the .pdf will be visible in Nextcloud OR you get the error which is the cause why the app doesn´t work

Second possibilty: The command fails. You get the error which is the cause why the app doesn´t work

I think I maybe got the bug in this app which prevents printing something useful into the log: In Converter.php where the command is built (line 92):

$exec = $command . $clParameters . escapeshellarg($tmpDir) . ' ' . escapeshellarg($tmpPath);

should be

$exec = $command . $clParameters . escapeshellarg($tmpDir) . ' ' . escapeshellarg($tmpPath).' 2>1&';

That redirects STDERR to STDOUT which is saved to $out. At least that´s what I found in the web; I never worked in php before.

[manu-p]

Hi, thanks @xDeerStalker for taking care.

I used a .odt instead of .docx, don't think it makes a big difference, and here's what the output gives me (sudoed as root):

# sudo -u www-data /usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/test.odt'
/usr/bin/libreoffice: 54: cd: can't cd to /root

and when I cd /tmp before:

# sudo -u www-data /usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/test.odt' 
javaldx failed!
Warning: failed to read path from javaldx
X11 connection rejected because of wrong authentication.

(process:18560): dconf-CRITICAL **: 10:15:32.096: unable to create directory '/root/.cache/dconf': Permission denied.  dconf will not work properly.
X11 connection rejected because of wrong authentication.

"2>1&" is just a shell redirection to put stderr (2) to stdout (1).

[xDeerStalker]

Yep, that's the same error i got, just with another home directory for www-data which seems to be /root in your case.

please verify with: cat /etc/passwd | grep www-data

this should in your case return something like this: www-data:x:33:33:www-data:/root:/usr/sbin/nologin

usually the home-directory of www-data is not /root but /var/www and the output should look like this: www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin

I would suggest changing the home directory, but you have to check if www-data owns something inside /root for whatever reason. To do so, type:

sudo ls -la /root | grep www-data

if the command above returns nothing you´re probably good to go.
Also, ensure root is the only user with access to /root by doing

sudo chmod 700 /root

If you didn´t revert the changes i suggested (creating /var/www/.cache/dconf and changing priviledges with chown) you can just sudo usermod -d /var/www www-data

and the bug should be solved.

In any case, don´t mess with priviledges for the /root directory and don´t create .cache there. At least that´s my advice. It can lead to serious disasters if www-data or anyone else who is not root can read .mysql_history or .bash_history of root due to a misconfiguration.

"2>1&" is just a shell redirection to put stderr (2) to stdout (1).

Yes and exec() in php seems to only print stdout which would be the reason why the exception you get when you run the command manually is not logged. A similar problem can be found here

[manu-p]

Yep, that's the same error i got, just with another home directory for www-data which seems to be /root in your case.

Nope!

please verify with: cat /etc/passwd | grep www-data

# grep www-data /etc/passwd
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin

Exactly what you're showing too.

There must be some other issue somewhere...

# ll /var/www
total 40
drwxr-xr-x  9 www-data www-data 4096 Sep 22 14:00 ./
drwxr-xr-x 13 root     root     4096 Apr  9 11:27 ../
drwxr-xr-x  3 www-data www-data 4096 Sep 15 11:00 .cache/
. . .

# ll -R /var/www/.cache/
/var/www/.cache/:
total 12
drwxr-xr-x 3 www-data www-data 4096 Sep 15 11:00 ./
drwxr-xr-x 9 www-data www-data 4096 Sep 22 14:00 ../
drwx------ 2 www-data www-data 4096 Sep 15 11:00 dconf/

/var/www/.cache/dconf:
total 12
drwx------ 2 www-data www-data 4096 Sep 15 11:00 ./
drwxr-xr-x 3 www-data www-data 4096 Sep 15 11:00 ../
-rw------- 1 www-data www-data    2 Sep 25 13:15 user

# file /var/www/.cache/dconf/user
/var/www/.cache/dconf/user: data
# od /var/www/.cache/dconf/user
0000000 000000
0000002

[xDeerStalker]

Yea, i´m stupid. I could reproduce your error and it´s no miracle - you got the wrong command from me =/

Ignore this: sudo -u www-data /usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/ConverterTest.docx'

and replace that step from https://github.com/nextcloud/workflow_pdf_converter/issues/118#issuecomment-698276354 with: 3: sudo su www-data -s /bin/bash and then run:

4: usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/ConverterTest.docx'

For whatever reason in this case, sudo -u www-data ; and sudo su www-data; ; don't produce the same output. Usually, any commands that are triggered by the command behind sudo (let´s say a lot of programs in a script) indirectly are run with the same user you sudoed into. This is not the case here. I think as soon as the exception throws and libreoffice exits with code != 0, sudo is considered done.

Then, when the actual error gets printed, you are back in the account you started from (root i guess?) and all the relative paths with their nice shortcuts are replaced by absolute paths - but this pathconversion should happen inside libreoffice before it exits and not somewhere else. Probably a bug of libreoffice ; i´m able to reproduce it from multiple accounts. It´s always requesting access to ~, ~/.cache and ~/.cache./dconf where ~ is replaced by the absolute path to the user-home of the user you sudo -u from.

Okay, long story short: sudo su directly into www-data and you probably get the exception that you have no access to /var/www. That was the last step from the initial answer https://github.com/nextcloud/workflow_pdf_converter/issues/118#issuecomment-695221354 which you didn´t execute.

So grant privileges ( chown www-data:www-data /var/www ). Test the command again from a www-data shell. If this works, test in Nextcloud - this should work now.

Now it gets funny: remove all the given the privileges again which means changing /var/www back to root:root and the same for .cache and .cache/dconf

It still works.

How?

black magic of open source software ;-)

I think the permissions for www-data/libreoffice aren´t really needed to get the job done but they are an initial requirement to get it working.

[phil16727]

Hey Guys, I am now totally confused sorry. I am on Ubuntu 20.04 installed Nextcloud as a snap. There was not /var/www/.cache/dconf folder as mkdir was noch able to create it. I did it by hand. So what I did: sudo mkdir /var/www/.cache/dconf/ sudo chown -R www-data:www-data /var/www/.cache sudo chown www-data:www-data /var/www sudo su www-data -s /bin/bash

NO Luck the Convert Workflow App is still not working. Is there an "how to" to get this thing running? I do get this error could not convert /admin/files/FileName, reason: []

[xDeerStalker]

@phil16727 after sudo su www-data -s /bin/bash you did usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/ConverterTest.docx' where /tmp/ConverterTest.docx is a valid .docx, owned by www-data and is readable by www-data?

While executing that command you don´t get any exception and after the command is finished and /tmp/ConverterTest.pdf still doesn´t exist, is that correct? However, if /tmp/ConverterTest.pdf exists, the app should work too because it does exactly the same thing.

Is there an "how to" to get this thing running?

I dont´t think so, maybe our conversation will lead to some "how to"

[phil16727]

Hi, well after I sudo su www-data -s /bin/bash i went back to the Nextcloud GUI and Uploaded a file (Upload is the trigger to convert a docx into a pdf) but still got could not convert /admin/files/FileName, reason: [] in the nextcloud logger

[xDeerStalker]

Okay, please cp /var/www/nextcloud/data/path/to/ConverterTest.docx /tmp

after that, run usr/bin/libreoffice --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir '/tmp' '/tmp/ConverterTest.docx' from the shell where you are www-data@hostname (the one where you entered sudo su www-data -s /bin/bash). In doubt, the command whoami should return www-data.

if that finishes without an error, verify /tmp/ConverterTest.pdf exists. If the command throws an error, please post it here

[phil16727]

Hey @xDeerStalker Thanks for the reply. Just to get this right, we are about to roll out the nextcloud to 25 User or so. I wont ssh every single file verytime it changes. Therefore I dont really get it, why I should convert a single docx by ssh?

[xDeerStalker]

because that's the only way to get the real cause of the error while the nextcloud log will always say "reason: []". You will not have to do that every time you want to generate a pdf, just now to get it debugged

[phil16727]

OK i will do that, but I have no var/www/nextcloud since the installtion is though snap. Any idea?

[xDeerStalker]

/var/snap/nextcloud/common/, from https://github.com/nextcloud/nextcloud-snap/wiki/Finding-files

[phil16727]

Ok, well I looked for the files I uploaded. They are located at /var/snap/nextcloud/common/nextcloud/data/admin/files But I cant get past common /var/snap/nextcloud/common/ bash: cd: nextcloud: Permission denied

Sorry for the newbee question how do i figure out thre Systemuser and HomeDrive of Nextcloud