Optional watchman backend?

[mystor]

When reading through the next ideas draft I noticed that one of the benefits of the new design is the ability to dynamically switch between backends as-needed, depending on system resources etc. As someone who wants to use notify as a nice plug & play backend for filesystem watching, that sounds like a great feature which reduces ergonomics burden & produces a better experience :tada:.

That being said, I often have other watcher tools already set up on my large projects, such as 'watchman'. Given the dynamic watcher model, I wonder if it would be possible/worthwhile to detect a running watchman instance, and use it as a cross-platform backend if it happens to already be watching the repository in question.

I'm not sure whether or not the new notify model supports backends being provided by external crates. If it does, this feature may make less sense to include in core, and could perhaps be implemented externally.

[passcod]

Absolutely! Watchman support has been on the next wishlist for a long time, possibly as a "blessed" or "official but external" backend.

[passcod]

Been thinking about this because it provides such a good story both for direct users of the library and for end-users of downstream applications.

So, specifically, to be in core, this requires:

• Multi-source, which was shipped in 5.0.0-pre.0 by switching to crossbeam. What this is about is that notify needs to be able to get events from multiple sources and send them all along a single channel. Crossbeam channels are mpmc, which supports this usecase.
• Unified events, which is a requirement of 5.0.0 to ship. That is, there should be no type difference between sources of events, like there is in 4.0 between debounced and immediate.
• Dynamic recommended backend, which was a next thing and is not currently on the feature list for implementation. The current recommendation is made statically at compile time. That's not great, not just for watchman, but also for other potential backends, such as fanotify on Linux, which requires root. Additionally, given watch limits in inotify on Linux, being able to detect that and fallback would be very strong. So that's definitely on the map, just not quite yet.

And it would be nice to have:

• Dynamic live backend. This is about notify switching from one backend to another in the middle of watching. It's about covering the case where notify is asked to provide a watch, selects watchman, and then at some point later the watchman socket fails, possibly because watchman went away. Notify can certainly error out at that point, but it would probably be a much better experience if it instead started native watches to replace the watchman ones.
• Dynamic live backend recovery. This is about going back to a preferred backend if it becomes available again later. For example, if watchman is taken down for a minute, we want to switch away from it to native events immediately to keep events flowing, but the best experience would also be to recover back to watchman as that is a more effective backend.

External backends are actually already implemented way back from the very start, because notify backends (currently) are implementations of a trait. The trait is public, and you can implement your own. It's not even particularly more awkward to use than the current experience.

So, in theory, writing an external watchman backend is entirely possible right now, and putting it in core requires one additional feature / contribution.