spacedrive
spacedrive copied to clipboard
spacedriveapp
[ENG-304] "Too many files" macOS unhappy with prod!
In production builds, if you index a location large enough, maybe 1000+ files, it will crash the core. The UI will stay in an unresponsive state until relaunched, at that point it crashes immediately, error is from macOS. Head boop for having too many open connections to files on the system. Not sure if we're forgetting to drop some context somehow or there's a glitch deeper down. Will require some devout debugger (such as @oscar @brendan or @maxichrome, or @ericson.ds999 if you have time). It is rather annoying and I haven't a clue where to start looking for a fix. Jamie Pine on Linear
Generally speaking on a OS level you "open" a file which is a syscall, the OS then returns a number called a file handle, file id(fid) or whatever you wanna call it, its a number. that number can then be used for read/write operations.
Important is that the file needs to be closed with the "close" call, dangling file handles remain open until the process exits. so its important to close these
@liz3 the issue is that there's so many places that the file descriptors could be left dangling, even in relation to something specific such as location indexing. it's possible that #468 resolved this issue but it hasn't been tested yet. i tried to look into it when this issue first came about, but didn't get very far as nothing really stuck out.
yes it can be tedious in big projects. One way of debugging such issues is breaking on the open/close syscalls, in a way where all locations of allocation are seen and then there can be a match where the close is missing.
Generaally it might be a idea in big projects to centralise file handles into a singleton service which then can make it a lot easier to allocate/track and ultimately also close file handles
It's also worth nothing that Rust should close the file descriptor for us when the file handle is dropped. I suspect this issue is more to do with us opening too many files at the same while running a job as opposed to us not dropping the the file description but I haven't look into it so I could be incorrect here.
