groupfolders
groupfolders copied to clipboard
nextcloud
Error "This file no longer exists" on group folder files after a timeout
How to use GitHub
- Please use the 👍 reaction to show that you are affected by the same issue.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Steps to reproduce
- This occurs on two of my three Nextcloud instances. I don't know how to make it happen on the other, although they are configured similarly (it has fewer files).
- In the web browser, surf down the tree in a group folder. (This does not happen on normal folders, but Collectives files have a problem as well.) I have some PDF files and image files, for instance.
- Wait a minute.
- Click on the file.
- A red error "This file no longer exists" appears.
Note: This appears to be a problem accessing the group file in the root directory (see the Nextcloud log error below). So, I theorized that I need to access the root directory in order to refresh the file cache. I verified this manually. However, I eventually created a script to access the root directory via webdav every 10 seconds, which does seem to solve the problem for group folders. (I had to access the Collectives directory as well in order to solve the problem for those files.) However, the cache seems to be on a per-user basis, so it seems I would have to create the same script (along with an app password) for every user account on the system, which will be a difficult task, not to mention a potential security problem for the users.
This problem does not appear for general user files -- but it does appear for group folders and Collectives folders.
I have another installation -- a test system with fewer files in group files. I can't seem to reproduce the problem on that system, though somewhat similar in the installation.
I assume I might have some setting wrong, but I'm not sure what that might be. However, the systems worked fine for quite some time. (The problem seems to have cropped up somewhat recently, but at first I didn't identify it clearly.) A few months ago I went through an update to PHP and added a bunch of recommended optimizations to all of my installations: https://docs.nextcloud.com/server/latest/admin_manual/installation/server_tuning.html
I'm using the default user mechanism (defined via Nextcloud) versus something like LDAP or some other SSO technology.
I tried 27.1.0RC2 but I don't think it solved the problem, and I reverted the system to a checkpoint before the RC2 installation.
This might not be a bug in group folders, but it seems to specifically manifest when using group folders. I'm sorry I don't have a more robust reproduction scenario.
Addition: My second production environment is apparently also vulnerable to this, but it seems to take longer to timeout. Additionally, I updated that one to 15.1.0 and experienced the fatal error, to which I applied the edit listed in #2548 ... The problem still persists after the update to 15.1.0
Expected behaviour
Normally the file would open. And they do open ... until I wait a while. Maybe 20 seconds?
Actual behaviour
If the web page is idle for a while (say 20 seconds) then the error will appear. The error can be fixed by viewing the root directory.
Server configuration
Operating system: Linux Mint 21.2
Web server: Apache
Database: MariaDB
PHP version: 8.2.10 Also tried latest 8.1 version with similar effect.
Nextcloud version: (see Nextcloud admin page) 27.0.2
Group folders version: 15.0.2
Updated from an older Nextcloud/ownCloud or fresh install: Updated. Been active for about a year, generally with updates to the latest build soon after release.
Where did you install Nextcloud from: Downloaded file.
Are you using external storage, if yes which one: local/s3/smb/sftp/... No
Are you using encryption: yes/no No (just https)
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/Saml/... No
Client configuration
Browser: Brave or Edge
Operating system: Windows
Logs
Web server error log
Web server error log
I can't find any error corresponding to this condition
Nextcloud log (data/nextcloud.log)
Nextcloud log
Debug webdav Sabre\DAV\Exception\NotFound: File with name //MCF could not be located at 2023-09-09T20:36:01+00:00
apps/dav/lib/Connector/Sabre/Directory.php line 227
0. .../sabre/dav/lib/DAV/Tree.php line 78
OCA\DAV\Connector\Sabre\Directory->getChild(
)
1. 3rdparty/sabre/dav/lib/DAV/Tree.php line 73
Sabre\DAV\Tree->getNodeForPath(
)
2. 3rdparty/sabre/dav/lib/DAV/Tree.php line 73
Sabre\DAV\Tree->getNodeForPath(
)
3. 3rdparty/sabre/dav/lib/DAV/Tree.php line 73
Sabre\DAV\Tree->getNodeForPath(
)
4. 3rdparty/sabre/dav/lib/DAV/Tree.php line 73
Sabre\DAV\Tree->getNodeForPath(
)
5. 3rdparty/sabre/dav/lib/DAV/Server.php line 971
Sabre\DAV\Tree->getNodeForPath(
)
7. .../sabre/dav/lib/DAV/Server.php line 1662
Sabre\DAV\Server->getPropertiesIteratorForPath(
)
8. 3rdparty/sabre/dav/lib/DAV/Server.php line 1647
Sabre\DAV\Server->writeMultiStatus(
)
9. .../sabre/dav/lib/DAV/CorePlugin.php line 346
Sabre\DAV\Server->generateMultiStatus(
)
10. .../sabre/event/lib/WildcardEmitterTrait.php line 89
Sabre\DAV\CorePlugin->httpPropFind(
)
11. 3rdparty/sabre/dav/lib/DAV/Server.php line 472
Sabre\DAV\Server->emit(
)
12. 3rdparty/sabre/dav/lib/DAV/Server.php line 253
Sabre\DAV\Server->invokeMethod(
)
13. 3rdparty/sabre/dav/lib/DAV/Server.php line 321
Sabre\DAV\Server->start(
)
14. apps/dav/lib/Server.php line 364
Sabre\DAV\Server->exec(
)
15. apps/dav/appinfo/v2/remote.php line 35
OCA\DAV\Server->exec(
)
16. remote.php line 172
require_once("\/var\/www\/nextcloud22\/apps\/dav\/appinfo\/v2\/remote.php")
Browser log
Browser log
It appears to show the file name (a .PDF file) and status 404 protocol h2 type xhr initiator xhrjs:220 size 291B time 41ms
I installed 27.1.0 with updates to all apps that I'm running. This problem still exists.
The fact that recent changes were regarding lazy updates leads me to believe that there is a bug in the non-lazy-izing of the lazy updates feature. This is just an educated guess, because the problem manifests itself as a problem of caching (which would require updates, but possibly could be refreshed in a "lazy" fashion).
I'm thinking that when I have time, I'm going to need to do a deep-dive on why this is happening (e.g. look at the source code, add log messages, etc.). In other words, I'm going to have to start becoming a Nextcloud developer. (Similar things have happened to me in the past.) If anyone has any tips on best practices, I'm all ears.
I have the suspicion that this is not just a group folder issue, but right now group folder usage is my driving factor and the one I know how to reproduce.
Thanks for the link.
It seems this has been happening with regular shares as well.
Quickly looking at the source, code, I traced things to public function getFileInfo($path, $includeMountPoints = true) which does reference a cache, so at least things are looking a bit like I expected. I wonder if I can come up with some kind of ugly workaround to force the cache to be refreshed when a file can't be found.
I think I may be able to use this function to send information to the log for debugging:
$this->logger->warning('Storage not valid for mountpoint: ' . $mount->getMountPoint(), ['app' => 'core']);
I could sprinkle a few of those into the getFileInfo() and getCacheEntry() functions in lib/private/Files/View.php.
It looks like the getCacheEntry() would fail if there is a lock exception. I wonder if that might be in play.
When retrieving a file successfully (this is an image file in a group folder subtree), getFileInfo() is called with a sequence of calls with full path down to "//MCF" and finally "/__groupfolders". In this case, the $mount->getMountPoint() is "/
When retrieving a file fails (after waiting a short timeout), the access cycle starts at the basic "//MCF" level (not at the full file subtree) and the $mount->getMountPoint() is simply "/
I'm not sure what makes it forget about the long path name, or to divide the path differently in each case. I'm also not sure why it divides the path differently.
Details:
Adding log messages within getFileInfo() like this:
if (!$data instanceof ICacheEntry) {
$this->logger->warning('No data: ' . $relativePath . ';' . $internalPath . ';' . $path . ';' . $this->fakeRoot . ';' . $mount->getMountPoint(), ['app' => 'core']);
return false;
}
$this->logger->warning('Yes data: ' . $relativePath . ';' . $internalPath . ';' . $path . ';' . $this->fakeRoot . ';' . $mount->getMountPoint(), ['app' => 'core']);
Results in the following:
Good access:
Warning core Yes data: /MCF/MCF Board Meetings/2023/MCF BOD 2023-07-08 2023-10-14T00:36:52+00:00
July-August;MCF Board Meetings/2023/MCF BOD 2023-07-08
July-August;/<username>/files/MCF/MCF Board Meetings/2023/MCF BOD
2023-07-08 July-August;/<username>/files;/<username>/files/MCF/
Warning core Yes data: /MCF/MCF Board Meetings/2023;MCF Board 2023-10-14T00:36:52+00:00
Meetings/2023;/<username>/files/MCF/MCF Board
Meetings/2023;/<username>/files;/<username>/files/MCF/
Warning core Yes data: /MCF/MCF Board Meetings;MCF Board 2023-10-14T00:36:52+00:00
Meetings;/<username>/files/MCF/MCF Board
Meetings;/<username>/files;/<username>/files/MCF/
Warning core Yes data: 2023-10-14T00:36:52+00:00
//MCF;;/<username>/files/MCF;/<username>/files;/<username>/files/MCF/
Warning core Yes data: /__groupfolders;__groupfolders;/__groupfolders;;/ 2023-10-14T00:36:52+00:00
Warning core Yes data: 2023-10-14T00:36:52+00:00
/appdata_xxxxxxxxxxxx/photos/geonames;appdata_xxxxxxxxxxxx/photos/geonames;/appdata_xxxxxxxxxxxx/photos/geonames;;/
Failed Access:
Warning core No data: 2023-10-14T00:37:29+00:00
//MCF;files/MCF;/<username>/files/MCF;/<username>/files;/<username>/
I'm seeing the same behaviour with Nextcloud 28.0.6. I had it also with earlier versions, but the timeout that seems to be reached for this problem to occur has been gradually decreasing over the years. Currently my groupfolder is nearly unusable. I only have to wait for about 5s and I get the File not found error. Refreshing the root folder in another browser tab, fixes it.. for about 5s.
I'm seeing this problem also using de Gnome Nautilus WebDav mapping where it gives or a file not found error, or it opens the file with only a part or no content, or Nautilus crashes. When Nautilus does not crash and I refresh the view when in this state, it refreshes to the root of the Nextcloud instance.
Even in de Android app I started having problems that after the timeout I get File not found errors or the view refreshes to the root of the instance..
Is anyone still working on this? I'm afraid I won't be able to, but I may be able to provide more debugging info?
@RobinR1 I'm running into this issue still on NC 30.0.4 I'm curious, are you serving NC with Apache or Nginx? If Apache, do you have any compression configured?
