nextcloud
[Bug]: Mac OS VFS: when fileproviderextdatabase.realm becomes large, Finder becomes unresponsive
⚠️ Before submitting, please verify the following: ⚠️
- [x] This is a bug, not a question or a configuration issue.
- [x] This issue is not already reported on Github (I've searched it).
- [x] Nextcloud Server and Desktop Client are up to date. See Server Maintenance and Release Schedule and Desktop Releases for supported versions.
- [x] I agree to follow Nextcloud's Code of Conduct
Bug description
The FileProviderExt process database seems to grow up very big ~/Library/Group Containers/com.nextcloud.desktopclient/FileProviderExt/Database/fileproviderextdatabase.realm. The size may be normal but this is causing a steady 100% CPU usage (one core) by the process, so either there's a "leak" that makes the database grow too big, or we need some more indexes / query optimization.
Steps to reproduce
In our organization, Finder (especially using client 3.14.1) becomes very slow on the NC folders soon after the first installation and configuration of the client: sometimes it takes a minute to open a folder or a file; this is always accompanied by a steadily high (100% of one core) CPU usage by the FileProviderExt process. Among the files opened by the process, I found out that ~/Library/Group Containers/com.nextcloud.desktopclient/FileProviderExt/Database/fileproviderextdatabase.realm which has grown quite big (over 100MB).
So I tried this:
- Disable Virtual File Sync in settings
- Quit the client,
- Clear Caches in ~/Library
- Remove the whole ~/Library/Group Containers/com.nextcloud.desktopclient folder.
- Reboot mac OS
- Re-Enable Virtual File Sync in settings
This made Finder become very responsive again, and the FIleProviderExt process does not eat up 100% CPU. Unfortunately I'm expecting the database to grow up again soon.
Expected behavior
I'd expect a much smaller database to be kept locally, or if not possible, the queries made not suffer that much from database size.
Which files are affected by this bug
The NC local share
Operating system
macOS
Which version of the operating system you are running.
Sonoma 14.x and Sequoia 15.0.1
Package
Official macOS 12+ universal pkg
Nextcloud Server version
29.0.8
Nextcloud Desktop Client version
3.14.1
Is this bug present after an update or on a fresh install?
Updated to a major version (ex. 3.3.6 to 3.4.0)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
Are you using an external user-backend?
- [x] Default internal user-backend
- [ ] LDAP/ Active Directory
- [ ] SSO - SAML
- [ ] Other
Nextcloud Server logs
No response
Additional info
No response
I am experiencing the same issue. Don't know, what is stored in that file-database, but the fun seems to start as soon as I start browsing folders on my Mac, which then change their icon from "not loaded" to "waiting for upload". Afaikt, the state doesn't change afterwards.
Maybe a spindump of the FileProviderExt would be a better choice to upload. I also regard this as a serious issue.
I've just installed Realm Studio and opened my ~100MB database.
- ItemMetadata counted 111k entries. At a first glance (I'm not RQL expert at all) I can't see duplicate lines, it rather seems to span on a big part of the file share, including stuff I don't work on, and presumably never do.
- LocalFileMetadata counts 61 entries, I guess they're the materialised files I am really working on.
I would like to share a thought about ItemMetadata (in case the current queries are considered good for the purpose and no bug is found): if the purpose of ItemMetadata is to avoid doing redundant IO, I'm not sure it is helpful to keep track of every item processed since the beginning of the client lifetime, because -at some point- it becomes slower than starting the db from scratch. I would consider using it more as a cache, i.e. keeping the items "touched" in the last X days only and removing the older stuff regularly, except for materialized items, to keep the count of ItemMetadata entries as small and useful as possible. The filesystem can change while the client is offline, so ItemMetadata can have large amounts of outdated entries. The cost of having them stored can therefore be higher than re-discovering them, and this is especially true for folders that have not been "touched" for a long time.
Thanks !
Did the same after you mentioned it. My 67MB database has approx. 98k metadata entries although I only browsedt through some folders in the Finder, not even opening more than one file. I wonder what this DB would be in size, if there was an account that had multiple groupfolders in it, which all would have a high number of files/folders.
I also noted, that while FileProviderExt was working, it was creating quite a huge amount of download traffic from the NC server.
Soo… I just wanted to see, how this works and I decided to wipe the FileProviderExt and start over while disabling the quick synchronization feature. I guessed that the algo was indiscriminately synchronizing probably 10% of the account's files metadata and I was curious, what would happen, if I disabled that. Turned out, that this time FileProviderExt tried to synchronize all file's metadata and them silently crashed somewhere along the way…
There was no notification and it was only me monitoring the FileProviderExt process itself, that made me notice that it crashed…
Also… the realm DB now consists of approx. 45k entries and has grown to 27MB in size. Doesn't that amount to approx. 512k per entry? How big is this DB supposed to grow? As stated earlier, we do plan to make heavy use of groupfolders, which will hold thousands of files each. Either the DB format needs to be optimized a lot, or this will probably not work for larger environments, which would be a real problem for us.
My preferred strategy is being as minimal as possible in terms of how much knowledge to keep locally. This also gives more guarantees on correctness. Ideally, only materialized items are important and even those can be wiped after X days of inactivity. My 2 cents.
Regards!
This is surely done that way to combat the performance shortcomings of WebDAV, but to me it also looks too aggressive. At least have an option to either size the amount of DB size allowed or maybe even better allow to choose different strategies for handling this metadata.
Also, there seems to be a bug in the FileProviderExt, which causes it to loop when it enters a folder with lots of files… I just tried to open a folder that contains 3583 files and the contents of the folder is not displayed, while the process for this particular FilesProviderExt continuously keeps busy at 99% CPU. I verified that on two clients and have taken a spindump from one, which I attached as well.
Thanks for your last message @budachst, from a user perspective seems like you better encircled the possible root cause of the performance issue. In our shared file structure I don't think we have a case with a so-high number of files in a single folder at the same level, but we reach the order of magnitude of a hundred for sure, in many folders.
This may explain why resetting the db gives immediate benefit (i.e. the provider won't sync those folders until newly discovered).
I am experiencing very similar symptoms on MacOS Sequoia 15.4.1 and NextCloud Client 3.16.3. Tried deleting various Caches, but to no avail. Sooner or later (especially on even slightly (!) slower connections (e.g. home vs work, not even talking about being on the train), the app just hangs and at least one of several FileProviderExt processes is stuck at around 100% CPU. NextCloud Client becomes unresponsive usually after a couple of minutes or at a random point in time (literally hard to tell, every once in a while it works for a couple of hours). Without killing the FileProviderExt processes, it's impossible to restart NextCloud.
Here's a potential cause for this issue:
I use Vectorworks, a BIM architecture software, with files easily in the size range of 100–500 MB. I save frequently (about once every 2–3 minutes) and Vectorworks creates for each save a SWAP file, deletes the original, then renames the SWAP. It seems that in VFS the MacOS Client somehow tries to synchronize all the deleted originals that are sent to the bin (vs just overwriting in the standard sync mode). So (it seems) I end up with easily several GB in the bin waiting to be uploaded and overwhelming the FileProviderExt.
The screenshot is for iCloud, but the same happens with NextCloud. Might very well be an Apple issue:
Can anyone try to reproduce this (e.g. keep creating and deleting 200MB+ files) and see if this can lead to NC VFS Client and/or FileProviderExt getting unresponsive? Note: this almost doesn't happen on very fast Internet connections (sync can "keep up"?).
Hi,
Not sure if it's related. We had a user that had issues with VFS on Mac OS where it got stuck for a long time with a "Loading" message when browsing trough files and folders making it unusable. Problem could be replicated on multiple devices and connecting with the same user account.
Turned out the user was added to a Team folder and then another user had also shared the same team folder with them using the regular share function. This resulted in that the folder showing up twice in the folder list (with the regular share having (2) at the end).
Removing the regular share solved the issue in this case.
Turned out the user was added to a Team folder and then another user had also shared the same team folder with them using the regular share function. This resulted in that the folder showing up twice in the folder list (with the regular share having (2) at the end).
Yeah… this will send the Desktop Client down the drain. We are actually, placing ACEs to our group shares' root to prevent that. So no more sharing the whole groupshare to anyone.
Interesting. I do have shares, and also almost everything is on external storages (SFTP). Works near-perfectly with standard sync on Mac, is quite buggy with VFS. Anyone see that experimental virtual files option in the standard shares on Mac? What does that @.: + 41 79 800 85 26On 26 Jun 2025, at 14:45, budachst @.> wrote:budachst left a comment (nextcloud/desktop#7326)
Turned out the user was added to a Team folder and then another user had also shared the same team folder with them using the regular share function. This resulted in that the folder showing up twice in the folder list (with the regular share having (2) at the end).
Yeah… this will send the Desktop Client down the drain. We are actually, placing ACEs to our group shares' root to prevent that. So no more sharing the whole groupshare to anyone.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>
Hi folks, 3.17.0 will contain several significant architectural changes that will eliminate the unresponsiveness caused by the database growing large. As soon as this version becomes available I encourage you to try it (preferably with a complete fresh set up -- which you can do by disabling VFS for the account, deleting the database, and re-enabling VFS) and to see if the issue is resolved. Please reopen this if you are still facing long load times
Hi @claucambra,
haven't had an issue with the database, but 3.17.0 shows the synching indication in the menubar forever - even when all of my 3 accounts show, that they are in sync. This will agitate people consoderably, I guess. I already removed the app completely and also removed /added all my acounts again, but to no avail.
My two cents: I've switched to the (experimental) placeholder approach on my Mac (have to enable experimental features by modifying the config file) and it's been working just fine for my four NC servers since a couple of weeks. It should be the recommended approach at least for the time being IMHO.