go-ceph icon indicating copy to clipboard operation
go-ceph copied to clipboard

monitor cephfs for changed/added files/directories

Open mzimmerman opened this issue 4 years ago • 4 comments

Is there any API that allows me to monitor or poll cephfs for a list of the changed/added files since X time? Currently, I walk the file system looking for those things, but that's expensive to the MDS as we have a lot of files and since I'm polling often, the MDS constantly has to keep that information in memory.

mzimmerman avatar Apr 13 '21 13:04 mzimmerman

Oh, like a sort of inotify kind of interface in libcephfs? I don't currently know of such a thing, and looking over the list of functions in libcephfs.h I don't see anything clearly related to that topic. I know for sure we don't have something like that in go-ceph.

It's certainly an interesting idea, so I encourage you to bring it up with the core ceph team. If they can suggest a specific api to use we can look to wrapping it in the future. If this is something they'd need to add, we could wrap it after it's been implemented in the C libraries.

Thanks!

phlogistonjohn avatar Apr 13 '21 14:04 phlogistonjohn

Hi,

Just checking if there has been any progress on this or if there's an upstream ticket that I can check?

darthShadow avatar May 11 '23 09:05 darthShadow

I don't know if @mzimmerman ever contacted ceph upstream about this. I don't remember seeing a tracker ticket for it or list email, but Ceph is large enough to easily miss stuff. If this is important to either of you please contact the cephfs dev team about it and feel free to CC me on the conversation - but since I'm pretty sure that no such API exists today so the conversation would need to start with Ceph.

phlogistonjohn avatar May 11 '23 22:05 phlogistonjohn

Raised a feature request: https://tracker.ceph.com/issues/62215

Please feel free to add any details I may have missed.

darthShadow avatar Jul 28 '23 05:07 darthShadow