cds-videos icon indicating copy to clipboard operation
cds-videos copied to clipboard

Fix/remove the broken FileInstance objects

Open switowski opened this issue 6 years ago • 5 comments

As reported by the email sent by checksum verification script (https://github.com/CERNDocumentServer/cds-videos/issues/1416), we have multiple FileInstance objects that are not linked to any Record or Bucket. We should understand why we have them in the DB and maybe remove them to make the email actually useful (8k lines in the email are not useful).

switowski avatar May 31 '18 14:05 switowski

A lot of FileInstance has URI empty, this is causing many exceptions in the tasks log:

File "/opt/cds/src/invenio-files-rest/invenio_files_rest/storage/pyfs.py", line 167, in pyfs_storage_factory
    assert default_location
AssertionError

These files should be probably removed.

ntarocco avatar Jun 07 '18 17:06 ntarocco

  • Still 42 FileInstances with ObjectVersion objects attached to them
  • Need to check 570 FileInstances with existing uri and size 0.

zzacharo avatar Jun 21 '18 09:06 zzacharo

After I removed all the FileInstance objects that are not connected to ObjectVersion, there are 18 new cases, so we are still creating those empty FileInstances. I can't really see what could be causing it, maybe users closing the tab right after starting the upload? So unless we figure out how to prevent this situation, I suggest we create another task (yay, the bibtasklets again!) or amend the file verification one to remove those empty FileIntance objects . Anyway, we should at some point create a task that will be cleaning things like this (yay, the inveniogc again!).

switowski avatar Jul 04 '18 16:07 switowski

One thing to remember when deleting FileInstance objects - make sure the creation date is like few days ago, so we don't delete those that belong to deposits that just started.

switowski avatar Jul 04 '18 16:07 switowski

@switowski @zzacharo this is linked with v1.0.6 which has been already released and deployed. Can we close it? if not, we should remove this milestone.

ludmilamarian avatar Jul 23 '18 14:07 ludmilamarian