server [Bug]: Maintenance:mode changes are not respected

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

occ maintenance:mode --on should force users to see the maintenance mode message

occ maintenance:mode --off should disable the maintenance mode message

In each case, occ status or grep maintenance config.php indicates that the status of the maintenance setting was changed as requested.

On my server, I need to execute systemctl restart php8.1-fpm before these changes take effect.

Steps to reproduce

Upgrade to NC27
Set maintenance mode on
Access the web interface
The system remains fully accessible unless I run systemctl restart php8.1-fpm

Expected behavior

If Maintenance mode is enabled, users should see a maintenance mode message
If Maintenance mode is disabled, users should be able to access the server

Installation method

Community Manual installation with Archive

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "192.168.1.7",
            "vcloud.aicr.org",
            "aicr.dyndns.org",
            "192.168.1.89"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded-for-headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/vcloud.aicr.org",
        "dbtype": "pgsql",
        "version": "27.0.0.8",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "theme": "",
        "maintenance": false,
        "forcessl": true,
        "forceSSLforSubdomains": true,
        "mail_smtpmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 2,
        "logtimezone": "Europe\/London",
        "log_authfailip": true,
        "log_rotate_size": 104857600,
        "cache_path": "",
        "knowledgebaseenabled": true,
        "enable_avatars": true,
        "allow_user_to_change_display_name": true,
        "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
        "enabledPreviewProviders": [
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\MP3",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\MSOfficeDoc",
            "OC\\Preview\\PDF",
            "OC\\Preview\\SVG",
            "OC\\Preview\\Photoshop"
        ],
        "asset-pipeline.enabled": true,
        "apps_paths": [
            {
                "path": "\/var\/www\/nextcloud\/apps",
                "url": "\/apps",
                "writable": true
            },
            {
                "path": "\/var\/www\/nextcloud\/apps-local",
                "url": "\/apps-local",
                "writable": false
            }
        ],
        "xframe_restriction": true,
        "mail_smtpport": "25",
        "updatechecker": false,
        "activity_expire_days": 3652,
        "enable_certificate_management": true,
        "trashbin_retention_obligation": "auto",
        "singleuser": false,
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": "0",
            "timeout": "0.0"
        },
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "mysql.utf8mb4": true,
        "updater.release.channel": "stable",
        "app_install_overwrite": [
            "passman",
            "files_mindmap"
        ],
        "mail_sendmailmode": "smtp",
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [
            "MFA"
        ],
        "twofactor_enforced_excluded_groups": []
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - announcementcenter: 6.6.1
  - bruteforcesettings: 2.7.0
  - circles: 27.0.0
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contactsinteraction: 1.8.0
  - dav: 1.27.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_mindmap: 0.0.28
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - impersonate: 1.14.0
  - integration_openai: 1.0.10
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - nextcloud_announcements: 1.16.0
  - notifications: 2.15.0
  - oauth2: 1.15.0
  - passman: 2.4.5
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_email: 2.7.3
  - twofactor_totp: 9.0.0
  - updatenotification: 1.17.0
  - viewer: 2.1.0
  - workflowengine: 2.9.0
Disabled:
  - dashboard: 7.7.0 (installed 7.0.0)
  - encryption: 2.15.0
  - files_retention: 1.15.0 (installed 1.15.0)
  - groupfolders: 14.0.2 (installed 14.0.2)
  - imap_auth_aicr: 0.1.3 (installed 0.1.3)
  - passwords: 2023.5.30 (installed 2023.5.30)
  - recommendations: 1.6.0 (installed 1.3.0)
  - registration: 2.1.0 (installed 2.1.0)
  - sharelisting: 1.1.1 (installed 1.1.1)
  - suspicious_login: 5.0.0
  - user_external: 3.1.0 (installed 3.1.0)
  - user_ldap: 1.17.0
  - user_status: 1.7.0 (installed 1.4.0)
  - weather_status: 1.7.0 (installed 1.4.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

I am seeing the same behavior on two servers.

Both servers use apache2 / PHP 8.1 / redis & postgresql

Jun 18 '23 13:06 mjmccarn

Check your FPM php.ini and make sure that:

opcache.validate_timestamps = 1
- https://www.php.net/manual/en/opcache.configuration.php#ini.opcache.validate-timestamps
opcache.revalidate_freq = X where X is something sane like 2 or 60
- https://www.php.net/manual/en/opcache.configuration.php#ini.opcache.revalidate-freq
Wait X seconds after initiating the maintenance command

That, or restart your PHP FPM as you've discovered. Optimizations have trade-offs. Just have to decide which ones you prefer. :)

Jun 18 '23 13:06 joshtrichards

Suspected duplicate of #37557

Jun 18 '23 14:06 joshtrichards

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

Jul 22 '23 00:07 nextcloud-command

Is it possible to clear or disable the opcache if the server detects that maintenance mode is active?

I am OK restarting php81-fpm to get out of maintenance mode, but going into or being in maintenance mode should be a mandatory condition that is not subject to other configuration settings.

Jul 29 '23 13:07 mjmccarn

Is it possible to clear or disable the opcache if the server detects that maintenance mode is active?

I am OK restarting php81-fpm to get out of maintenance mode, but going into or being in maintenance mode should be a mandatory condition that is not subject to other configuration settings.

I'd also be interested in triggering a redis update when the server is in maintenance mode, as I'm using redis for caching and have a similar issue.

Sep 04 '23 09:09 jessebot

@mjmccarn Not in the way that would be helpful at present unfortunately. Consider that the PHP opcache isn't shared between the CLI and web-app server (mod_php/fpm) processes. So even though we actually do try to invalidate the opcache in our code, it's irrelevant since invalidating the CLI opcache doesn't change anything for the running web app... https://github.com/nextcloud/server/issues/37557#issuecomment-1596168246 (and elsewhere).

P.S. I'm going to close this issue and consolidate further tracking/discussion to: #37557 and #39079 in the interest of not spreading these discussions out everywhere. I encourage you to subscribe to those issues to track progress and anyone else that sees this issue to give those referenced issues a subscribe and/or :+1: if impacted.

@jessebot Can you elaborate? The PHP opcache and the data/memcaches (such as Redis) don't have much to do with each other. If you're having a similar issue, it's due to the PHP opcache too (and your installation's PHP opcache settings such as opcache.validate_timestamps and opcache.revalidate_freq), but likely not your memcache (redis).

Sep 10 '23 13:09 joshtrichards

Duplicate of #37557

Sep 10 '23 13:09 joshtrichards

server server copied to clipboard

[Bug]: Maintenance:mode changes are not respected

⚠️ This issue respects the following points: ⚠️

Bug description

Steps to reproduce

Expected behavior

Installation method

Nextcloud Server version

Operating system

PHP engine version

Web server

Database engine version

Is this bug present after an update or on a fresh install?

Are you using the Nextcloud Server Encryption module?

What user-backends are you using?

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

server
server copied to clipboard