files_antivirus icon indicating copy to clipboard operation
files_antivirus copied to clipboard

Optimize query for unscanned files

Open zero0cool0 opened this issue 1 year ago • 1 comments

If my interpretation of the code in BackgroundScanner is correct, getNodeForFile() relies on the existence of an existing mount to determine the Node for a given fileId.

Sometimes, this fails. Specifically, this line is returning an empty array. $cachedMounts = $this->userMountCache->getMountsForFileId($fileId)

Tracking this further, I found that no corresponding mount can be found for some fileId that are present in oc_filecache when joining oc_mounts through oc_storage.

Why is this even a problem, other than creating many log entries?

If the number of such entries in oc_filecache is large then the entire batch contains non-scannable entries, starving the scanning of other entries in oc_filecache that have a corresponding mount.

My PR proposes to only process files whose fileId that can be related to an existing mount.

More details: I have a lot of entries in oc_filecache that look like this:

MariaDB [nextcloud]> select fileid,storage,path,size from oc_filecache where fileid=1044856;
+---------+---------+---------------------------------------------------------------+------+
| fileid  | storage | path                                                          | size |
+---------+---------+---------------------------------------------------------------+------+
| 1044856 |      30 | appdata_ocp617nvaw61/preview/2/7/b/3/b/c/9/667010/256-154.png | 7476 |
+---------+---------+---------------------------------------------------------------+------+

These entries correctly relate to an entry in oc_storages:

+------------+----------------------------+-----------+--------------+
| numeric_id | id                         | available | last_checked |
+------------+----------------------------+-----------+--------------+
|         30 | local::/var/www/html/data/ |         1 |         NULL |
+------------+----------------------------+-----------+--------------+

However, no mount can be found:

MariaDB [nextcloud]> select * from oc_mounts where storage_id=30;
Empty set (0.000 sec)

zero0cool0 avatar Jul 29 '24 16:07 zero0cool0

Hello there, Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

github-actions[bot] avatar Aug 13 '24 02:08 github-actions[bot]