[Bug]: Group Folders App is auto-disabled during upgrade and causes data loss

[sonaro]

⚠️ This issue respects the following points: ⚠️

• [X] This is a bug, not a question or a configuration/webserver/proxy issue.
• [X] This issue is not already reported on Github (I've searched it).
• [X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• [X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• [X] I agree to follow Nextcloud's Code of Conduct.

Bug description

During a major version upgrade, the Group Folders app was disabled for an unknown reason (needed an update?). While this app was disabled, and after Maintenance mode was turned off, any clients that had access to those group folders and that auto re-sync'd DELETED the client side group folders to match their new nextcloud server permissions since the GF app was disabled.

This is a major issue, for three primary reasons:

• all of the group folders on the client, including any new unsync'd data saved to the client group folders were DELETED permanently and could not be recovered (unless the client had a local, separate backup running concurrently with nextcloud that also includes the Nextcloud local folder/group folders)
• server-side backups were not able to recover lost data for any data that was not fully uploaded yet --- for instance, if files are written after maintenance started, or if quota was temporarily exceeded
• wasted resources in having to re-download all of the content in the server group folders once the group folders issue was resolved

Upgrade history: -- 20.0.13 --> 20.0.14.2 --> 21.0.9.1 --> 22.2.10.2 --> 23.0.10.1 --> 24.0.6.1 -- backups were made between each version upgrade. Was not able to find the point when Group Folder app was disabled.

Steps to reproduce

This test plan also includes the client-side data loss example.

NC Client: 3.3.6 on OSX 12.4 (21F79) NC Server: 20.0.13, upgraded to 24.0.6.1 (Not sure when Group Folder app was disabled, but have all logs)

1. Turn on Maintenance Mode
2. Save a new test file to a group folder on a client filesystem that will be sync'd
3. Successfully update from a previous major release
4. Turn off Maintenance Mode
5. Check status of Group Folders App. If it says "Disabled", this bug applies
6. Wait for the client that no longer has access to the group folders to sync and it will delete the group folders, the group folder sync'd data, and and the unsync'd test file

To resolve access: 7. Group Folders app then needs to be manually updated and enabled via the web dashboard to restore access. Client data not sync'd cannot be restored (unless client has a separate concurrent backup)

Expected behavior

The expected behavior is to have the client side app "unmanage" the group folder and allow the entire folder to persist, rather than delete it.

Or:

• "unmanage" and not delete any content in the group folder but never uploaded yet (only existing copy on the client side).
• move the un-uploaded data to a different folder that the person can manually merge later on (not advised since client software may be saving to that location automatically)
• at a minimum, immediate action could be to add a data loss warning any time the Group Folders app is disabled

Installation method

Community Web installer on a VPS or web space

Operating system

Debian/Ubuntu

PHP engine version

PHP 7.4

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Updated to a major version (ex. 22.2.3 to 23.0.1)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• [X] Default user-backend (database)
• [ ] LDAP/ Active Directory
• [ ] SSO - SAML
• [ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.DOMAIN.TLD"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/cloud.DOMAIN.TLD",
        "dbtype": "mysql",
        "version": "24.0.6.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "dbindex": 0,
            "timeout": 1.5
        },
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpsecure": "ssl",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "theme": "",
        "loglevel": 0,
        "updater.release.channel": "stable",
        "mail_sendmailmode": "smtp",
        "app_install_overwrite": [
            "groupfolders",
            "passman"
        ],
        "encryption.legacy_format_support": false,
        "encryption.key_storage_migrated": false,
        "mysql.utf8mb4": true,
        "versions_retention_obligation": "auto",
        "twofactor_enforced": "true",
        "default_phone_region": "+1",
        "twofactor_enforced_groups": [],
        "twofactor_enforced_excluded_groups": [],
        "updater.secret": "***REMOVED SENSITIVE VALUE***"
    }
}

List of activated Apps

NOTE:  this is the status of the system with "Group Folders" working properly (after the app was manually Enabled again)

Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - admin_audit: 1.14.0
  - analytics: 4.4.0
  - bruteforcesettings: 2.4.0
  - calendar: 3.5.2
  - checksum: 1.1.5
  - circles: 24.0.1
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contacts: 4.2.2
  - contactsinteraction: 1.5.0
  - dashboard: 7.4.0
  - dav: 1.22.0
  - drawio: 1.0.3
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_external: 1.16.1
  - files_markdown: 2.3.6
  - files_pdfviewer: 2.5.0
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - forms: 2.5.1
  - groupfolders: 12.0.2
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - maps: 0.2.1
  - metadata: 0.17.0
  - nextcloud_announcements: 1.13.0
  - notes: 4.5.1
  - notifications: 2.12.1
  - oauth2: 1.12.0
  - onlyoffice: 7.5.6
  - ownpad: 0.6.18
  - password_policy: 1.14.0
  - photos: 1.6.0
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - quota_warning: 1.15.0
  - recommendations: 1.3.0
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - support: 1.7.0
  - survey_client: 1.12.0
  - systemtags: 1.14.0
  - tasks: 0.14.5
  - text: 3.5.1
  - theming: 1.15.0
  - twofactor_backupcodes: 1.13.0
  - twofactor_totp: 6.4.1
  - unsplash: 2.0.1
  - updatenotification: 1.14.0
  - user_ldap: 1.14.1
  - user_status: 1.4.0
  - viewer: 1.8.0
  - weather_status: 1.4.0
  - workflowengine: 2.6.0
Disabled:
  - deck: 1.7.2
  - emlviewer: 1.0.2
  - encryption: 2.7.0
  - external: 4.0.0
  - files_accesscontrol: 1.14.1
  - files_automatedtagging: 1.14.0
  - limit_login_to_ip: 3.1.0
  - passman: 2.1.5
  - ransomware_detection: 0.10.0
  - richdocuments: 6.3.1
  - spreed: 14.0.5
  - twofactor_u2f: 6.3.1
  - user_saml: 5.0.3

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Logs are huge and also contain sensitive data.  Please ask and I can provide relevant sections.

Additional info

I'm not yet able to determine exactly when Group Folders app was disabled, so any info on how to best filter through the logs would be most helpful.