pullcord
pullcord copied to clipboard
Partial/distributed grabs are hard to do
Pullcord is currently not well suited for grabbing only a select portion of the history. It is possible to start grabbing from some point in time which is not the beginning, but to do that you'd need to specifically prepare a log file with a fake entry, which seems hacky. Getting logs only up to a certain point is impossible to do reliably right now.
Adding support for partial grabs is almost certainly going to increase the amount of command line options. I'd prefer to avoid clutter and/or feature creep, but I'm not sure if it's possible with the way the application is laid out right now. It wasn't really designed with large scale grabs in mind.
Implementing #1 could be a "good enough" solution, since channel logs even for large, active servers seem to be at most a few GiB per channel.
If you grab single channels with -c
, each channel in their own separate parent directory, you end up with multiple copies of the guild log. To get a single file, we could make a tool to merge guild logs which would filter out duplicate entries.