sahlberg
[kodi] New files inaccessible until reboot - Overzealous file/directory caching?
I recently made a post on the Kodi forums about a bug I've been experiencing with files I host on my NFS server.
Kodi Forums - Newly added files in previously-accessed directories inaccessible until reboot
I have Kodi+Libreelec on an ODROID-C2, I have my NFS server and my Kodi Database on an ODROID-XU4 (running Arch Linux ARM), and I have another Kodi instance (also set up to use the database) on my laptop. I use the laptop Kodi instance to do my rescanning and database related stuff. All devices are running Kodi 17.
Steps to reproduce:
- On the ODROID-C2, play a random episode from [current season] of [show] (optionally: immediately stop playback), and keep the menu in the [current season]'s listing
- add [new episode] to [current season]'s folder of [show] on the NFS storage (via the laptop, or some other machine on the LAN)
- on the laptop, open Kodi and run "Update Library"
- on the ODROID-C2, go "back" from the [current season] view to the list of seasons, then back into [current season]
- attempt to "play" the newly added episode entry
- receive "This file is no longer available" error, despite the file definitely actually being present on the server (if you press "i", on my skin at least, the correct full path for the file is displayed)
Kodi uses libnfs for its NFS support. As I detail in the thread, my suspicion is that libnfs is caching the files it sees in a given folder on the first access it makes, but then refuses to ever update the entries in that cache, even when new files have been added to the folder, and when the Kodi database now contains entries pointing to these new files.
I was suggested to open a bug here on the libnfs tracker.
If there's any further info I can provide, please do not hesitate to ask.
Hi, sorry must have missed this email.
Libnfs caches directory content based on the mtime timestamp for the directory. This check is done every time the application opens the directory using the opendir() call.
Can you compile libnfs for your system? If so you can try replacing this line https://github.com/sahlberg/libnfs/blob/master/lib/nfs_v3.c#L2911 with:
cached = NULL;
and thus disable directory caching completely.
On Tue, Jun 5, 2018 at 1:47 AM, Adam Hirst [email protected] wrote:
I recently made a post on the Kodi forums about a bug I've been experiencing with files I host on my NFS server.
Kodi Forums - Newly added files in previously-accessed directories inaccessible until reboot https://forum.kodi.tv/showthread.php?tid=332282
I have Kodi+Libreelec on an ODROID-C2, I have my NFS server and my Kodi Database on an ODROID-XU4 (running Arch Linux ARM), and I have another Kodi instance (also set up to use the database) on my laptop. I use the laptop Kodi instance to do my rescanning and database related stuff. All devices are running Kodi 17.
Steps to reproduce:
- On the ODROID-C2, play a random episode from [current season] of [show] (optionally: immediately stop playback), and keep the menu in the [current season]'s listing
- add [new episode] to [current season]'s folder of [show] on the NFS storage (via the laptop, or some other machine on the LAN)
- on the laptop, open Kodi and run "Update Library"
- on the ODROID-C2, go "back" from the [current season] view to the list of seasons, then back into [current season]
- attempt to "play" the newly added episode entry
- receive "This file is no longer available" error, despite the file definitely actually being present on the server (if you press "i", on my skin at least, the correct full path for the file is displayed)
Kodi uses libnfs for its NFS support. As I detail in the thread, my suspicion is that libnfs is caching the files it sees in a given folder on the first access it makes, but then refuses to ever update the entries in that cache, even when new files have been added to the folder, and when the Kodi database now contains entries pointing to these new files.
I was suggested to open a bug here on the libnfs tracker.
If there's any further info I can provide, please do not hesitate to ask.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/sahlberg/libnfs/issues/257, or mute the thread https://github.com/notifications/unsubscribe-auth/AAeNkOJ6RMhUL7grFjkgo2L8QkLgYI4Qks5t5VZ4gaJpZM4UZViI .
UUID=386cb49b-6b69-41bd-ab8c-111af6cb0fd1 /mnt/Odin xfs defaults,nofail,relatime 0 2 -snip- /mnt/Odin /srv/nfs/Odin none bind,nofail 0 2
You are accessing the files via /srv/nfs/Odin which is a bind mount. That should not make any difference but could you see if this effect also show up if you access via the real mountpoint /mnt/Odin ?
Thanks for responding, @sahlberg.
Since I primarily run this on machines running LibreELEC, I'd need to look into how to go about building and installing my own compiled version of libnfs there.
The Kodi instance which does the database update seems able to view the new files just fine (presumably since the act of database updating flushes these caches, or otherwise forces "true" rescans of directories), which rules out my main laptop as being the testbed for a modified build of libnfs, where it would be most convenient. I do however have a 2nd laptop running Linux, onto which I can install libnfs+kodi, first verify the same behaviour as with Kodi+LibreELEC, then apply the libnfs modifications.
From your description of the issue, it sounds like one of the following would likely address the issue:
- the directory mtime should get changed whenever new files are added to it
- libnfs should not necessarily trust "directory mtime hasn't changed" as being indicative of "directory does not contain new files"
- kodi should not necessarily trust libnfs' claims that files are not present, when the kodi database claims there should be one - performing a rescan (another call of
opendir()?)
I was under the impression that using a bind mount was "the correct way" to handle NFS mounts. I can try modifying this temporarily, but it will have to be at a later date, when the NFS server is less likely to be used.