warcit icon indicating copy to clipboard operation
warcit copied to clipboard

Appends .csv to the log file string

Open Shrinks99 opened this issue 1 year ago • 2 comments

This will ensure that log files are read in the proper format and that I stop opening it in my text editor! ;)

Changes

  • Appends .csv to the user input string passed through the --log argument.
  • Updates help text

Shrinks99 avatar Nov 03 '23 02:11 Shrinks99

Other thing to consider: Should we just expect the full filename with the extension to the --log argument? And if there's an issue with another tool that's using this library and isn't doing that, we could handle it upstream.

Something to think about. Sometimes a little less "magic" ends up being more flexible for a wide variety of use cases (and a little less logic to handle).

tw4l avatar Nov 13 '23 17:11 tw4l

My sense is that auto-appending file extensions to provided names only makes sense if the same base name is used for several output files. For instance, if a crawler would ask for a "collection name" and then produce a filename.warc.gz, filename.cdxj, and filename.log or similar.

despens avatar Nov 13 '23 19:11 despens