server icon indicating copy to clipboard operation
server copied to clipboard
verified publisher icon nextcloud

[Bug]: Versions still appear in the Files view after running `php occ files_versions:cleanup`

Open elpraga opened this issue 2 years ago • 15 comments

⚠️ This issue respects the following points: ⚠️

Bug description

I may have run into a problem.

By mistake, I have manually deleted old versions directly in the Nextcloud data directory directly on the server.

Then, after reading about the correct solution, I ran php occ files_versions:cleanup myuser.

I was under the assumption that after this step, old versions will no longer appear in the Files app in the web interface.

They do appear, and obviously, they are inaccessible.

Is this the expected behavior and can this situation on my server be fixed?

Thank you for your reply and advice!

P.S. I've marked the issue as a bug because I believe that after running php occ files_versions:cleanup myuser, the old versions should not appear regardless of my previous error.

I did run occ files:scan myuser .

Steps to reproduce

  1. run php occ files_versions:cleanup myuser
  2. old versions are still visible

Expected behavior

old versions are no longer visible after running php occ files_versions:cleanup myuser (regardless if they had physically existed on disk or not)

Installation method

None

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

  • [X] Default user-backend (database)
  • [ ] LDAP/ Active Directory
  • [ ] SSO - SAML
  • [ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "filelocking.enabled": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "elpraga.no-ip.biz",
            "cloud.tomaskaluza.net",
            "192.168.1.146",
            "raspberry",
            "127.0.0.1",
            "192.168.0.10"
        ],
        "enable_previews": true,
        "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
        "preview_office_cl_parameters": " --headless --nologo --nofirststartwizard --invisible --norestore --convert-to pdf --outdir \/tmp ",
        "preview_max_x": 1920,
        "preview_max_y": 1080,
        "enabledPreviewProviders": {
            "0": "OC\\Preview\\PNG",
            "1": "OC\\Preview\\JPEG",
            "2": "OC\\Preview\\GIF",
            "3": "OC\\Preview\\Image",
            "5": "OC\\Preview\\Epub",
            "6": "OC\\Preview\\OpenDocument",
            "7": "OC\\Preview\\Font",
            "11": "OC\\Preview\\Illustrator",
            "12": "OC\\Preview\\Postscript",
            "13": "OC\\Preview\\Photoshop",
            "14": "OC\\Preview\\TIFF",
            "15": "OC\\Preview\\MarkDown",
            "16": "OC\\Preview\\Movie",
            "17": "OC\\Preview\\Font",
            "18": "OC\\Preview\\PDF"
        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "version": "27.0.0.8",
        "installed": true,
        "default_phone_region": "ES",
        "auth.bruteforce.protection.enabled": true,
        "appstoreenabled": true,
        "apps_paths": [
            {
                "path": "\/var\/www\/cloud.tomaskaluza.net\/apps",
                "url": "\/apps",
                "writable": true
            }
        ],
        "forcessl": true,
        "theme": "",
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "forceSSLforSubdomains": true,
        "appstore.experimental.enabled": true,
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 0,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "minimum.supported.desktop.version": "2.0.0",
        "quota_include_external_storage": false,
        "overwrite.cli.url": "https:\/\/cloud.tomaskaluza.net",
        "mail_smtpauthtype": "PLAIN",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpport": "465",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "htaccess.RewriteBase": "\/",
        "dbtype": "mysql",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "beta",
        "mysql.utf8mb4": true,
        "app_install_overwrite": [
            "files_texteditor",
            "richdocumentscode_arm64",
            "facerecognition",
            "health",
            "breezedark"
        ],
        "versions_retention_obligation": "1 1",
        "has_rebuilt_cache": true,
        "memories.exiftool_no_local": true,
        "memories.ffmpeg_path": "\/usr\/bin\/ffmpeg",
        "memories.ffprobe_path": "\/usr\/bin\/ffprobe",
        "memories.no_transcode": false,
        "memories.transcoder": "\/var\/www\/cloud.tomaskaluza.net\/apps\/memories\/exiftool-bin\/go-vod-aarch64",
        "memories.qsv": false,
        "memories.vod.path": "\/var\/www\/cloud.tomaskaluza.net\/apps\/memories\/exiftool-bin\/go-vod-aarch64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - analytics: 4.9.4
  - bookmarks: 13.0.1
  - breezedark: 26.0.0
  - bruteforcesettings: 2.7.0
  - calendar: 4.4.2
  - circles: 27.0.0
  - cloud_federation_api: 1.10.0
  - collectives: 2.5.0
  - comments: 1.17.0
  - contacts: 5.3.2
  - contactsinteraction: 1.8.0
  - cookbook: 0.10.2
  - cospend: 1.5.10
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.10.0
  - event_update_notification: 2.2.0
  - external: 5.2.0
  - facerecognition: 0.9.20
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_automatedtagging: 1.17.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_texteditor: 2.15.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - forms: 3.3.1
  - gpxpod: 5.0.12
  - groupfolders: 15.0.0
  - health: 1.6.2
  - imageconverter: 1.3.5
  - impersonate: 1.14.0
  - integration_deepl: 1.1.0
  - integration_giphy: 1.0.6
  - integration_google: 2.0.2
  - integration_openai: 1.0.10
  - integration_replicate: 1.0.6
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - mail: 3.2.2
  - memories: 5.2.0-beta.1
  - music: 1.8.4
  - news: 22.0.0-beta2
  - nextcloud_announcements: 1.16.0
  - notes: 4.8.0
  - notifications: 2.15.0
  - oauth2: 1.15.0
  - password_policy: 1.17.0
  - passwords: 2023.7.30-build4960
  - photos: 2.3.0
  - polls: 5.1.0
  - previewgenerator: 5.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recognize: 4.2.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - richdocuments: 8.1.0
  - richdocumentscode_arm64: 23.5.5
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - social: 0.6.1
  - spreed: 17.0.1
  - support: 1.10.0
  - survey_client: 1.15.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - tables: 0.5.1
  - tasks: 0.15.0
  - text: 3.8.0
  - text_templates: 1.0.3
  - theming: 2.2.0
  - theming_customcss: 1.14.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_nextcloud_notification: 3.7.0
  - twofactor_totp: 9.0.0
  - updatenotification: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflow_pdf_converter: 1.12.0
  - workflow_script: 1.12.0
  - workflowengine: 2.9.0
Disabled:
  - apporder: 0.15.0 (installed 0.15.0)
  - audioplayer: 3.4.0 (installed 3.4.0)
  - audioplayer_editor: 0.3.0 (installed 0.3.0)
  - camerarawpreviews: 0.8.1 (installed 0.8.1)
  - checksum: 1.2.2 (installed 1.2.2)
  - cms_pico: 1.1.0-beta.1 (installed 1.0.19)
  - data_request: 1.14.0 (installed 1.14.0)
  - drawio: 2.1.1 (installed 2.1.1)
  - duplicatefinder: 0.0.15 (installed 0.0.15)
  - encryption: 2.15.0
  - epubreader: 1.4.7 (installed 1.4.7)
  - extract: 1.3.5 (installed 1.3.5)
  - files_3d: 0.5.0 (installed 0.5.0)
  - files_markdown: 2.4.1 (installed 2.4.1)
  - files_photospheres: 1.26.4 (installed 1.26.4)
  - files_readmemd: 2.0.0 (installed 2.0.0)
  - files_retention: 1.16.0 (installed 1.16.0)
  - gestion: 2.3.3 (installed 2.3.3)
  - google_synchronization: 1.0.9 (installed 1.0.9)
  - gpxedit: 0.0.14 (installed 0.0.14)
  - group_everyone: 0.1.13 (installed 0.1.13)
  - issuetemplate: 0.7.0 (installed 0.7.0)
  - maps: 1.0.2 (installed 1.0.2)
  - metadata: 0.18.0 (installed 0.18.0)
  - nextbackup: 21.1.0 (installed 21.1.0)
  - ocsms: 2.2.0 (installed 2.2.0)
  - onlyoffice: 8.1.0 (installed 8.1.0)
  - phonetrack: 0.7.6 (installed 0.7.6)
  - quicknotes: 0.8.10 (installed 0.8.10)
  - radio: 1.0.3 (installed 1.0.3)
  - scanner: 0.2.3 (installed 0.2.3)
  - sharerenamer: 3.2.0 (installed 3.2.0)
  - talk_simple_poll: 1.3.1 (installed 1.3.1)
  - timemanager: 0.3.7 (installed 0.3.7)
  - timetracker: 0.0.79 (installed 0.0.79)
  - transfer: 0.6.1 (installed 0.6.1)
  - user_ldap: 1.17.0
  - user_usage_report: 1.11.0 (installed 1.11.0)
  - video_converter: 1.0.5 (installed 1.0.5)
  - weather: 1.7.7 (installed 1.7.7)
  - whiteboard: 0.0.3 (installed 0.0.3)

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

elpraga avatar Jun 28 '23 15:06 elpraga

@szaimen I've noticed that you added the feedback label.

Does it mean you need more feedback from me?

elpraga avatar Jul 05 '23 10:07 elpraga

cc @artonge

szaimen avatar Jul 10 '23 13:07 szaimen

@elpraga no it means that this issue is feedback about a recent feature :)

artonge avatar Jul 10 '23 13:07 artonge

Is there a chance this behaviour will be manded @artonge ?

elpraga avatar Jul 17 '23 16:07 elpraga

Can you check your Nextcloud log and pull the stack trace or error returned when you attempt to access these bogus versions?

joshtrichards avatar Aug 27 '23 00:08 joshtrichards

Can you run the occ file:scan command and see if they are still visible ?

artonge avatar Sep 21 '23 14:09 artonge

I have this issue too and file:scan and file:cleanup doesn't fix it, I use a hetzner storage share and they saw after I deleted the files they were still showing in the database I also found this https://github.com/nextcloud/server/issues/37461 issue and I would suspect its related although for this its in reverse, it is not removing it in the database it needs to once the files are deleted through OCC or expire through the retention setting but it works if i do a manual delete in the webui

Nypheena avatar Oct 24 '23 17:10 Nypheena

Hello, I have the same issue. I deleted the old versions in the data directory directly (because I did not need them any longer) after a migration. And now the old versions still appear in the UI. The UI also becomes laggy when trying to access the Versions tab for such a file. I was able to deleted the old/inaccessible versions for this file manually from the UI and the UI stopped lagging. old_versions

Seems that the old versions are still referenced somewhere. The database I imagine. Is there a way to clean these references safely from the database?

Anonym777 avatar Nov 12 '23 10:11 Anonym777

Our installation appears to be suffering from the same ptoblem. Not caused by manual deletion, but possibly problems with external storage. The effect is as described though, and I would be interested in how to remove these stale database entries referring to version files that do not physically exist.

ruedigerkupper avatar Dec 09 '23 20:12 ruedigerkupper

I was not able to reproduce. Is the issue still occurring? Could one of you list some reproduction steps to get into such state?

artonge avatar Jan 25 '24 16:01 artonge

@artonge

I was not able to reproduce. Is the issue still occurring? Could one of you list some reproduction steps to get into such state?

What version you on? Because I do not believe anything has changed with v27

As for reproducing, all you gotta do is upload a file twice so it gets versioned then run the occ command to clear out all versions and you’ll notice while the files on disk is removed the database still have them so they’ll show in the versions list on the uploaded file

for me this also applies with retention settings when auto cleared it also doesn’t clear from its database so it always shows all versions that ever existed just the version file itself no longer exists on disk

EDIT: I just tested on my hetzner storage share which runs 27.1.5 and the issue remains, the versions all appear on the list for the file but they are deleted on disk, database is not cleared of them

Nypheena avatar Jan 25 '24 19:01 Nypheena

I can reproduce what @Zairhn did on Nextcloud v28.0.1.1

justsomescripts avatar Jan 29 '24 09:01 justsomescripts

@justsomescripts @Zairhn I was not able to reproduce. Can you check if there is anything related in your Nextcloud server logs? You can check the log file just after running the command. Can you confirm that the command is occ versions:expire and not occ files_versions:cleanup?

artonge avatar Jan 30 '24 11:01 artonge

@artonge

@justsomescripts @Zairhn I was not able to reproduce. Can you check if there is anything related in your Nextcloud server logs? You can check the log file just after running the command. Can you confirm that the command is occ versions:expire and not occ files_versions:cleanup?

the command is not versions:expire it is infact cleanup as the title of this bug report states, but I have had the same issue with versions:expire, I also don't have log access on hetzner storage share

Nypheena avatar Jan 30 '24 19:01 Nypheena

Hello, could one of you test this PR: https://github.com/nextcloud/server/pull/43263 It adds a step to flush the files_versions table before deleting the files_versions folder.

artonge avatar Feb 01 '24 15:02 artonge