Better handling of big log files to avoid crash of the viewer

[arnowelzel]

Steps to reproduce

1. Set loglevel to in config/config.php to 0 ('loglevel' => 0).
2. Execute many actions which will trigger a log message to get written.
3. Open the log viewer when the logfile has more than 60000 entires.

Expected behaviour

The log viewer displays the log.

If the log is too big to be displayed, truncate to the newest n entries (where n is a value which can be handled by the frontend - for example 1000 or 5000) or paginate the log.

Actual behaviour

The log viewer crashes, so the page will get blank. It's not clear when exactly this happens. I have seen this with a logfile which counts about 62000 entries, but I believe the actual limit is lower and also depends on what browser and runtime environment you use.

Server configuration

Operating system: Ubuntu 20.04

Web server: Apache 2.4.41-4ubuntu3.10

Database: MariaDB 10.5.15

PHP version: 7.4

Nextcloud version: 24.0.0

Where did you install Nextcloud from: nextcloud.com

List of activated apps:

Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - admin_audit: 1.14.0
  - apporder: 0.15.0
  - bookmarks: 10.3.1
  - calendar: 3.3.0
  - circles: 24.0.0
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contacts: 4.1.0
  - contactsinteraction: 1.5.0
  - dashboard: 7.4.0
  - dav: 1.22.0
  - deck: 1.7.0
  - drawio: 1.0.2
  - external: 4.0.0
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_fulltextsearch: 24.0.0
  - files_fulltextsearch_tesseract: 24.0.0
  - files_pdfviewer: 2.5.0
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - forms: 2.5.0
  - fulltextsearch: 24.0.0
  - fulltextsearch_elasticsearch: 24.0.0
  - gpxpod: 4.3.0
  - keeweb: 0.6.9
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - nextcloud_announcements: 1.13.0
  - notes: 4.3.1
  - notifications: 2.12.0
  - oauth2: 1.12.0
  - onlyoffice: 7.3.4
  - password_policy: 1.14.0
  - phonetrack: 0.7.0
  - photos: 1.6.0
  - polls: 3.6.1
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - quota_warning: 1.14.0
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - spreed: 14.0.1
  - support: 1.7.0
  - survey_client: 1.12.0
  - systemtags: 1.14.0
  - tasks: 0.14.4
  - text: 3.5.1
  - theming: 1.15.0
  - theming_customcss: 1.11.0
  - twofactor_backupcodes: 1.13.0
  - updatenotification: 1.14.0
  - user_status: 1.4.0
  - viewer: 1.8.0
  - workflowengine: 2.6.0
Disabled:
  - encryption: 2.2.0
  - files_external: 1.15.0
  - recommendations: 0.4.0
  - user_ldap
  - weather_status: 1.3.0

Nextcloud configuration:

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.0x0c.de"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "skeletondirectory": "",
        "overwrite.cli.url": "https:\/\/nextcloud.0x0c.de",
        "dbtype": "mysql",
        "version": "24.0.0.12",
        "installed": true,
        "htaccess.RewriteBase": "\/",
        "maintenance": false,
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "knowledgebaseenabled": false,
        "theme": "",
        "simpleSignUpLink.shown": false,
        "loglevel": 2,
        "log_authfailip": true,
        "mysql.utf8mb4": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_smtpauthtype": "LOGIN",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "trashbin_retention_obligation": "auto, 30",
        "app_install_overwrite": [
            "apporder",
            "deck",
            "keeweb",
            "appointments",
            "drawio",
            "bookmarks",
            "spreed"
        ],
        "has_rebuilt_cache": true,
        "onlyoffice": {
            "verify_peer_off": true
        },
        "default_language": "de",
        "default_phone_region": "DE",
        "check_data_directory_permissions": false,
        "log_rotate_size": 52428800
    }
}

Client configuration

Browser: Vivaldi 5.2.2623.46 (Stable channel) (64-Bit)

Operating system: Windows 10

Logs

Nextcloud log (data/owncloud.log)

(not relevant)

Browser log

react-dom.production.min.js:216 Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at t.findDOMNode (react-dom.production.min.js:289:284)
    at t.transition (CSSTransitionGroupChild.js:129:30)
    at t.r.componentWillEnter (CSSTransitionGroupChild.js:97:15)
    at t.s.performEnter (TransitionGroup.js:83:19)
    at TransitionGroup.js:192:21
    at Array.forEach (<anonymous>)
    at t.componentDidUpdate (TransitionGroup.js:191:17)
    at co (react-dom.production.min.js:219:502)
    at xl (react-dom.production.min.js:259:160)
    at t.unstable_runWithPriority (scheduler.production.min.js:18:343)

Uncaught (in promise) Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at t.findDOMNode (react-dom.production.min.js:289:284)
    at t.transition (CSSTransitionGroupChild.js:129:30)
    at t.r.componentWillEnter (CSSTransitionGroupChild.js:97:15)
    at t.s.performEnter (TransitionGroup.js:83:19)
    at TransitionGroup.js:192:21
    at Array.forEach (<anonymous>)
    at t.componentDidUpdate (TransitionGroup.js:191:17)
    at co (react-dom.production.min.js:219:502)
    at xl (react-dom.production.min.js:259:160)
    at t.unstable_runWithPriority (scheduler.production.min.js:18:343)

[adriangalbincea]

I have the same issue, logging page is not loading anything, just spinning forever...

[danir-de]

We can trigger the issue when changing log settings or enabling auto refresh.

[crazycorndog]

I had the very same issue, logging page was not loading anything, just spinning forever...

Nextcloud 24(.0.4)

the logfile was 24M big, with 4554 lines:

-rw-r----- 1 nextcloud nextcloud 24M 18. Aug 00:18 /var/log/nextcloud/nextcloud.log
cat /var/log/nextcloud/nextcloud.log | wc -l
4554

I renamed the original logfile and touch'ed and chown'ed a new logfile, now the logging page is loading just fine (and saying that there are no log entries yet) but the logger doesnt seem to write anything to the new logfile now, even with debug loglevel enabled, the file is still empty

edit: while executing occ maintenance:mimetype:update-js an error happened in background and got logged in the log file, but still no debug level entries or alike

[strugee]

Note: the underlying React error message is Unable to find node on an unmounted component. (mostly adding this so that if folks look up the full React error string and then search GitHub for it, this issue will show up).

[luxifr]

same here with

www-data@2d5e879532e7:~/html/data$ ls -lh nextcloud.log
-rw-r----- 1 www-data www-data 3.0M Jan  1 02:39 nextcloud.log
www-data@2d5e879532e7:~/html/data$ wc -l nextcloud.log
2752 nextcloud.log

[Antreesy]

Quickly checked at Nextcloud 28 with 10k+ rows in a logfile, requests to logfile are chunked by 50 rows,

• no visible freezes with 3k lines shown.
• local file opened with 10k lines - quite slow, but manageable

Frontend handling can be improved with virtual scroller 🚀

https://www.npmjs.com/package/vue-virtual-scroller