nextcloudpi icon indicating copy to clipboard operation
nextcloudpi copied to clipboard

nc-previews 3 out of 4 processes stop generating previews

Open sserkez opened this issue 5 years ago • 1 comments

Hi, when executed on rock64, 3 out of 4 processes stop generating previews.

Terminal output

2019-12-29T18:37:01+00:00 Scanning folder /<username>/files/Media/Photos/2019
2019-12-29T18:37:13+00:00 Scanning folder /<username>/files/Media/Photos/2019/2019                                     .01_France
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 In Folder.php line 162:
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00   [OCP\Files\NotPermittedException]
2019-12-29T18:40:05+00:00   Could not create folder
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 Exception trace:
2019-12-29T18:40:05+00:00  () at /var/www/nextcloud/lib/private/Files/Node/Folde                                     r.php:162
2019-12-29T18:40:05+00:00  OC\Files\Node\Folder->newFolder() at /var/www/nextclo                                     ud/lib/private/Files/AppData/AppData.php:155
2019-12-29T18:40:05+00:00  OC\Files\AppData\AppData->newFolder() at /var/www/nex                                     tcloud/lib/private/Preview/Generator.php:419
2019-12-29T18:40:05+00:00  OC\Preview\Generator->getPreviewFolder() at /var/www/                                     nextcloud/lib/private/Preview/Generator.php:115
2019-12-29T18:40:05+00:00  OC\Preview\Generator->getPreview() at /var/www/nextcl                                     oud/lib/private/PreviewManager.php:205
2019-12-29T18:40:05+00:00  OC\PreviewManager->getPreview() at /var/www/nextcloud                                     /apps/previewgenerator/lib/Command/Generate.php:184
2019-12-29T18:40:05+00:00  OCA\PreviewGenerator\Command\Generate->parseFile() at                                      /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:171
2019-12-29T18:40:05+00:00  OCA\PreviewGenerator\Command\Generate->parseFolder()                                      at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:169
2019-12-29T18:40:05+00:00  OCA\PreviewGenerator\Command\Generate->parseFolder()                                      at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:145
2019-12-29T18:40:05+00:00  OCA\PreviewGenerator\Command\Generate->generatePathPr                                     eviews() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:11                                     5
2019-12-29T18:40:05+00:00  OCA\PreviewGenerator\Command\Generate->execute() at /                                     var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:255
2019-12-29T18:40:05+00:00  Symfony\Component\Console\Command\Command->run() at /                                     var/www/nextcloud/3rdparty/symfony/console/Application.php:901
2019-12-29T18:40:05+00:00  Symfony\Component\Console\Application->doRunCommand()                                      at /var/www/nextcloud/3rdparty/symfony/console/Application.php:262
2019-12-29T18:40:05+00:00  Symfony\Component\Console\Application->doRun() at /va                                     r/www/nextcloud/3rdparty/symfony/console/Application.php:145
2019-12-29T18:40:05+00:00  Symfony\Component\Console\Application->run() at /var/                                     www/nextcloud/lib/private/Console/Application.php:213
2019-12-29T18:40:05+00:00  OC\Console\Application->run() at /var/www/nextcloud/c                                     onsole.php:97
2019-12-29T18:40:05+00:00  require_once() at /var/www/nextcloud/occ:11
2019-12-29T18:40:05+00:00
2019-12-29T18:40:05+00:00 preview:generate-all [-p|--path [PATH]] [--] [<user_id                                     >]
2019-12-29T18:40:05+00:00
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 In Folder.php line 162:
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00   [OCP\Files\NotPermittedException]
2019-12-29T18:40:34+00:00   Could not create folder
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 Exception trace:
2019-12-29T18:40:34+00:00  () at /var/www/nextcloud/lib/private/Files/Node/Folde                                     r.php:162
2019-12-29T18:40:34+00:00  OC\Files\Node\Folder->newFolder() at /var/www/nextclo                                     ud/lib/private/Files/AppData/AppData.php:155
2019-12-29T18:40:34+00:00  OC\Files\AppData\AppData->newFolder() at /var/www/nex                                     tcloud/lib/private/Preview/Generator.php:419
2019-12-29T18:40:34+00:00  OC\Preview\Generator->getPreviewFolder() at /var/www/                                     nextcloud/lib/private/Preview/Generator.php:115
2019-12-29T18:40:34+00:00  OC\Preview\Generator->getPreview() at /var/www/nextcl                                     oud/lib/private/PreviewManager.php:205
2019-12-29T18:40:34+00:00  OC\PreviewManager->getPreview() at /var/www/nextcloud                                     /apps/previewgenerator/lib/Command/Generate.php:184
2019-12-29T18:40:34+00:00  OCA\PreviewGenerator\Command\Generate->parseFile() at                                      /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:171
2019-12-29T18:40:34+00:00  OCA\PreviewGenerator\Command\Generate->parseFolder()                                      at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:169
2019-12-29T18:40:34+00:00  OCA\PreviewGenerator\Command\Generate->parseFolder()                                      at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:145
2019-12-29T18:40:34+00:00  OCA\PreviewGenerator\Command\Generate->generatePathPr                                     eviews() at /var/www/nextcloud/apps/previewgenerator/lib/Command/Generate.php:11                                     5
2019-12-29T18:40:34+00:00  OCA\PreviewGenerator\Command\Generate->execute() at /                                     var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:255
2019-12-29T18:40:34+00:00  Symfony\Component\Console\Command\Command->run() at /                                     var/www/nextcloud/3rdparty/symfony/console/Application.php:901
2019-12-29T18:40:34+00:00  Symfony\Component\Console\Application->doRunCommand()                                      at /var/www/nextcloud/3rdparty/symfony/console/Application.php:262
2019-12-29T18:40:34+00:00  Symfony\Component\Console\Application->doRun() at /va                                     r/www/nextcloud/3rdparty/symfony/console/Application.php:145
2019-12-29T18:40:34+00:00  Symfony\Component\Console\Application->run() at /var/                                     www/nextcloud/lib/private/Console/Application.php:213
2019-12-29T18:40:34+00:00  OC\Console\Application->run() at /var/www/nextcloud/c                                     onsole.php:97
2019-12-29T18:40:34+00:00  require_once() at /var/www/nextcloud/occ:11
2019-12-29T18:40:34+00:00
2019-12-29T18:40:34+00:00 preview:generate-all [-p|--path [PATH]] [--] [<user_id                                     >]
...

the last 4th process continues to load cpu and apparently yield new previews: visible in htop as /bin/bash /usr/local/bin/ncc preview:generate-all -n -v -p /<username>/files/Media/Photos/2019

btw, in ncp-report one should probably consider marking "trusted_domains" also as sensitive value

System information

NextCloudPi diagnostics

NextCloudPi version  v1.20.1
distribution         Debian Buster with Armbian Linux \l
automount            yes
USB devices          sda sdb
datadir              /media/myCloudDrive/ncdata
data in SD           no
data filesystem      btrfs
data disk usage      38G/224G
rootfs usage         4.9G/29G
swapfile             /var/swap
dbdir                /media/myCloudDrive/ncdatabase
Nextcloud check      ok
Nextcloud version    16.0.4.1
HTTPD service        up
PHP service          up
MariaDB service      up
Redis service        up
Postfix service      up
internet check       ok
port check 80        open
port check 443       open
IP                   ***REMOVED SENSITIVE VALUE***
gateway              ***REMOVED SENSITIVE VALUE***
interface            eth0
certificates         ***REMOVED SENSITIVE VALUE***
NAT loopback         no
uptime               10days

Nextcloud configuration

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",

        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "16.0.4.1",
        "overwrite.cli.url": "https:\/\/cloud.serkez.xyz\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "tempdirectory": "\/media\/myCloudDrive\/ncdata\/tmp",
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "maintenance": false,
        "loglevel": "2",
        "log_type": "file",
        "logfile": "\/media\/myCloudDrive\/ncdata\/nextcloud.log",
        "jpeg_quality": "60",
        "theme": "",
        "data-fingerprint": "7c6b1dc19ed02a8b3f0cdae2ece076bc",
        "htaccess.RewriteBase": "\/",
        "preview_max_y": "2048",
        "preview_max_x": "2048",
        "mail_sendmailmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***"
    }
}

HTTPd logs


Database logs


Nextcloud logs

{"reqId":"XgfcSzejx852qgDc@yTbEwAAAA0","level":2,"time":"2019-12-28T22:50:54+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/apps\/spreed\/api\/v1\/signaling\/settings","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgfcUDejx852qgDc@yTbFAAAAA8","level":2,"time":"2019-12-28T22:51:04+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/cloud\/capabilities","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgiQFCqN63hPs7kDeAyN3wAAAAU","level":2,"time":"2019-12-29T11:37:58+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/apps\/spreed\/api\/v1\/signaling\/settings","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgiQFCqN63hPs7kDeAyN3gAAAAM","level":2,"time":"2019-12-29T11:37:58+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/cloud\/capabilities","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"XgiQGCqN63hPs7kDeAyN4AAAAAc","level":2,"time":"2019-12-29T11:38:02+00:00","remoteAddr":"2.203.242.150","user":"--","app":"core","method":"GET","url":"\/ocs\/v2.php\/cloud\/capabilities","message":"Login failed: 'svitozar' (Remote IP: '2.203.242.150')","userAgent":"Mozilla\/5.0 (Android) Nextcloud-Talk v7.0.8","version":"16.0.4.1"}
{"reqId":"hn8aE8LVIbsx3y2j2ZMq","level":3,"time":"2019-12-29T18:37:24+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"unlink(\/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343637\/455-256.jpg): No such file or directory at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#228","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:37:47+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:37:56+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"hn8aE8LVIbsx3y2j2ZMq","level":3,"time":"2019-12-29T18:38:24+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"unlink(\/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343649\/256-144.jpg): No such file or directory at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#228","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:38:26+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:08+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:12+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:22+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"8uHalDzuMptTpOBxU2yS","level":3,"time":"2019-12-29T18:39:31+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"filesize(): stat failed for \/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343659\/455-256.jpg at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#169","userAgent":"--","version":"16.0.4.1"}
{"reqId":"13AJoMTQq9pOKz6Y87o4","level":2,"time":"2019-12-29T18:39:31+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:39:35+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":3,"time":"2019-12-29T18:39:53+00:00","remoteAddr":"","user":"--","app":"PHP","method":"","url":"--","message":"stat(): stat failed for \/media\/myCloudDrive\/ncdata\/appdata_ocmkxn4qrngc\/preview\/2343664\/256-256-crop.jpg at \/var\/www\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#143","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:39:53+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"hn8aE8LVIbsx3y2j2ZMq","level":2,"time":"2019-12-29T18:40:05+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}
{"reqId":"jTLouJSOleSGWCiwFz9Q","level":2,"time":"2019-12-29T18:40:10+00:00","remoteAddr":"","user":"--","app":"music","method":"","url":"--","message":"FileHooks::updated triggered for a non-existing file","userAgent":"--","version":"16.0.4.1"}

sserkez avatar Dec 29 '19 19:12 sserkez

btw, in ncp-report one should probably consider marking "trusted_domains" also as sensitive value

very good point. You should report this to nextcloud/server

nachoparker avatar Jan 19 '20 03:01 nachoparker