barnard59 icon indicating copy to clipboard operation
barnard59 copied to clipboard

Archetypes for file processing

Open ktk opened this issue 6 years ago • 0 comments

I've seen so far the following types of configs:

  • One mapping file for a CSV: CSV Filename can be provided in the mapping file
  • Two or more mapping files for one CSV: Same as above
  • One mapping file for multiple input files: In this scenario a certain mapping gets processed for multiple files with the same structure. A typical feature needed here is to somehow extract part of the filename (for example a date) and incorporate it into the mapped data as well.

More?

My point is that we should have default steps that can handle these different kind of mappings and all the user has to do is to parametrize them correctly (like input file location).

ktk avatar Nov 03 '19 16:11 ktk