Report an error when events are dropped or send events synchronously

[hnakamur]

Hi, first of all, thanks for a nice software!

The code at https://github.com/rjeczalik/notify/blob/5dd6205716539662f8f14ab513552b41eab69d5d/watchpoint.go#L88-L92 says it drop events when the receiver is too slow.

I want to know whether or not events are dropped by checking an error, not by looking at debug prints. To tell the truth, I think it's easier if events are sent synchronously.

For example, github.com/fsnotify/fsnotify uses an synchronous sending. https://github.com/fsnotify/fsnotify/blob/3c39c22b2c7b0516d5f2553f1608e5d13cb19053/inotify.go#L266

[rjeczalik]

@hnakamur There's definitely room for improvement here. If you're expecting high volume of events you may find QueuedWatch helpful - https://gist.github.com/rjeczalik/7faa93691e6da35b1f84, and eventually do coalescing of the events in your app. Relevant thread: https://github.com/rjeczalik/notify/issues/85

Leaving open to keep track of this feature.

[hnakamur]

@rjeczalik Thanks for your reply. I read thread #85 and understand the problem better now. Usually an event producer can cope with a consumer and slow the pace. However filesystem events occurs independently on event listeners. So it is unavoidable for events are dropped generally.

I think your QueuedWatch helps in some cases, but there is still possibility for events to be dropped. So I think we need error reporting mechanism.

Another way to reduce possibility of drop is filtering events before sending. It does not help if you are interested in all events though.

However, for example, my use case is writing a BSD tail like library in go. I want to know the following events.

1. The target file is created.
2. The target file is written.
3. The target file is renamed. I want to know the filename after rename too.
4. The target file is removed.

For 1 and 3, I need to watch the directory of the target file, but I am not interested other files in that directory than the target file.

I think it's better that events of uninterested file are not sent so that it can reduce the possibility of dropping events of interested files.

Currently you can specify interested event types in Watch function. https://godoc.org/github.com/rjeczalik/notify#Watch

How about changing this to func Watch(path string, c chan<- EventInfo, filter func(ei EventInfo) bool, events ...Event) error?

[rjeczalik]

How about changing this to func Watch(path string, c chan<- EventInfo, filter func(ei EventInfo) bool, events ...Event) error

Unfortunately we can't just change the API. Instead we could have utility functions to do what you want, which for now could live outside this repository. Once we extend the API to enable more features they may be merged back in.

For your particular use-case you could write a Filter helper function that could do what you want:

func Filter(c chan EventInfo, pred func(EventInfo) bool) <-chan EventInfo {
  filteredC := make(chan EventInfo, cap(c))

  go func() {
    for ei := range c {
      if pred(ei) {
        filteredC <- ei
      }
    }
  }()

  return filteredC
}

func FilteredWatch(path string, pred func(EventInfo) bool, events ...Event) (<-chan EventInfo, error) {
  c := make(chan EventInfo, defaultCap)

  if err := Watch(path, c, events...); err != nil {
    return nil, err
  }

  return Filter(c, pred), nil
}

If you would want to Stop filteredC you should also take care of revmap yourself :-)

[hnakamur]

I understand we cannot change the API now. Thanks for Filter and FilterWatch function! It is very helpful.