CopyFiles should support --watch

[johncrim]

I hope this is the right place to post this issue - I'm currently using Wyam and haven't tested Statiq, but from a quick glance at the code the CopyFiles module is essentially the same. This is a feature request I suppose, so I think it belongs in statiq.

Given this pipeline:

Pipelines.Add("Assets",
  CopyFiles("../../../assets/fonts/**/*").To((src) => "fonts/" + src.Path.FileName),
  CopyFiles("../../../assets/images/site/**/*").To((src) => "img/" + src.Path.FileName),
  CopyFiles("../../../assets/images/shared/**/*").To((src) => "img/" + src.Path.FileName)
);

The file copying works as expected, but I can't figure out a way to make it work while developing the site in --watch mode. I end up having to either manually copy the files to /output, or have to stop and re-start the watch mode on every change.

Obviously I'm copying files from outside the /input directory, but I think that should be supported.

I'd be willing to contribute an implementation for this - I assume the hard part would be turning each copyfiles path into a FileSystemWatcher + a filter.

[daveaglick]

Great feature request, in fact I think there's another similar one open from a while back in the Statiq.Web repo. Right now we just watch the input folders to catch adds, deletes, and updates with the assumption that most activity will happen there. In your case where some files are coming from non-input folder, we have a few options:

• Allow specifying additional non-input paths to watch.
• Watch any folder from which non-input files are read.
• Watch all non-input files that were previously read for changes.

The first option is probably the least error-prone and easiest to implement and would catch new files as well as update/delete, but also requires manual configuration. The second could cast too wide a net, but does watch for new files as well as update/delete. The third option is the most focused but would miss new files if those should be watched for.

I'm not sure which is the best choice - any thoughts?

[johncrim]

My preference, given a shallow understanding of the architecture, is option 2 - "watch all folders containing files which are read". Ideally there would be a way for modules to filter changes that are irrelevant, eg CopyFiles("../../assets/**/*.svg") wouldn't trigger a rebuild if a non SVG file was updated.

IMO option 3 is out, because, at least for the CopyFiles module, if a new file is added that matches the path, it should be copied.

I think the correct behavior when a file is deleted is ambiguous, and I would tend toward doing nothing given the module name "CopyFiles". If the name was SyncFiles then the delete should propagate.

My understanding is that file watching is efficient on Windows, at least for local file systems. The biggest problem I could see related to casting too wide a net is unnecessary full rebuilds based on changes to irrelevant files, which is why I think modules would need a way to filter changes that should trigger a rebuild. If this already exists (sorry, I don't know if it does), then this approach seems best.

I think if the "watch all folders containing source files, and ignore changes to irrelevant source files" idea is too difficult to implement, then adding paths to watch could work. But this approach doesn't seem ideal, due to requiring duplication in the config code.

[SilentSin]

I also like the sound of option 2 best. If it casts too wide a net, maybe you could have another fluent method to call after To() that can enable or disable this feature after you define the default setting in the global config file.