Beholder take 45s to watch a dir

[whatacold]

Hi,

Just noticed a weird thing, that beholder takes 45s to watch a dir on my pc, but it almost instantly finishes the job on another dir, is it normal?

;;; below dirs are on different fs

(time (beholder/watch prn "/home/ken/tmp/"))
;; "Elapsed time: 43874.502804 msecs"

(time (beholder/watch prn "/tmp/"))
;; "Elapsed time: 148.206494 msecs"

In the meantime, I'm trying to figure it out using clj-async-profiler.

[whatacold]

I found out from a flamegraph that this slowness is due to walking through the whole directory. I'm not sure if this is a problem, or it's just necessary to do that to watch a dir.

[rafd]

The underlying library hashes every file to dedupe events. Your directory either has a lot of files or some very large files, taking 45s to make those hashes.

It is possible to turn off this behaviour in the underlying library, but, beholder does not expose this functionality.

In the meantime, if this is a critical issue, you could tweak the create fn in beholder as described in the directory-watcher docs - you could fork beholder and use your own version in your project, or just copy beholder's single namespace into your project, or use with-redefs to replace the implementation of beholder's create when invoking beholder.

...or submit a patch to beholder to expose this functionality.

[jeroenvandijk]

I had a similar problem (#8 also has a solution for this) and I adapted the namespace a bit with the following create function:

(defn create
  "Creates a watcher taking a callback function `cb` that will be invoked
  whenever a file in one of the `paths` chages.

  Not meant to be called directly but use `watch` or `watch-blocking` instead."
  [cb paths & {:keys [file-hasher ^FileTreeVisitor visitor]}]
  (let [^FileTreeVisitor file-hasher
        (case file-hasher
          (nil true :default) FileHasher/DEFAULT_FILE_HASHER
          :last-modified-time FileHasher/LAST_MODIFIED_TIME
          false nil)]
    (-> (cond-> (DirectoryWatcher/builder)
          visitor
          (.fileTreeVisitor visitor)
          file-hasher
          (.fileHasher file-hasher)
          (false? file-hasher)
          (.fileHashing false))
        (.paths (map to-path paths))
        (.listener (fn->listener cb))
        (.build))))

And I call it like:

(doto (beholder/create callback [path] :file-hasher :last-modified-time)
          (.watchAsync))

I'm using it on a new Mac and it is much faster, and I think reliable, but I am still testing things. Not 100% about no problems with duplicate events.