ConduitIO
Connector: directory connector [Source/Destination]
Resource name
directory
Resource link
https://en.wikipedia.org/wiki/Directory_(computing)
Connector type
Source
Expected connector behavior
We already have the file connector, so this is the logical next step. Just point to a directory and let conduit do its job.
Usecases:
- Backups mainly
- People that use something like obsidian, and want to backup all files into S3
- Move files all files from an S3 bucket into a folder.
I was looking into golang recursive directory watchers recently, so here's some info that might be helpful for this
fsnotify is probably the most popular golang filewatchers, but doesnt do recursive watching. https://github.com/fsnotify/fsnotify/issues/18
frankenphp (a fantastic new golang-based php webserver, based in caddy) added such capabilities recently. They started with a PR from a contributor, then rolled their own golang watcher, then ended up using e-dant/watcher (which is also mentioned in the fsnotify thread above. Here's where they discovered e-dant and continued the discussion about and implementation of it. the e-dant dev seems to be quite happy to help others integrate it - you can see him communicating in both the fsnotify and frankenphp threads
Hope this helps
Oh @nickchomey hello! I was thinking about fsnotify aswell, sad that doesn't include adding new files. I remember reading somewhere that air also had good filewatching capabilities.
I'd like to do this connector soon at some point, looks very fun to program!
FYI I've added this (and other connector requests) to our internal connectors' project where we prioritize all connector requests. We'll let you know once we figure when we're going to work on this.
