barnard59
barnard59 copied to clipboard
Archetypes for file processing
I've seen so far the following types of configs:
- One mapping file for a CSV: CSV Filename can be provided in the mapping file
- Two or more mapping files for one CSV: Same as above
- One mapping file for multiple input files: In this scenario a certain mapping gets processed for multiple files with the same structure. A typical feature needed here is to somehow extract part of the filename (for example a date) and incorporate it into the mapped data as well.
More?
My point is that we should have default steps that can handle these different kind of mappings and all the user has to do is to parametrize them correctly (like input file location).