Streamlining list of files for some stages

[hiker]

Once #280 is merged, I want to get rid of the many differences places from which files for compilation etc are being picked up. When I needed to introduce a new step before running PSyclone, I found it very hard to identify from which artefact stores to pick up files :(

First issue: preprocessing Fortran files will copy .f90 files into the build directory, but does not add it to any artefact. So a new step has no easy way of finding all Fortran files!

Follow up example (analysis)

DEFAULT_SOURCE_GETTER = CollectionConcat([
    SuffixFilter('all_source', '.f90'),
    'preprocessed_c',
    'preprocessed_fortran',

    # todo: this is lfric stuff so might be better placed elsewhere
    SuffixFilter('psyclone_output', '.f90'),
    'preprocessed_psyclone',  # todo: this is no longer a collection, remove
    'configurator_output',
])

So this picks up a mixture of files from all_source (the aforementioned .f90 files that are copied but not added to an artefact), then the two preprocessed ones, then psyclone output files, and config output. Imho these files should be all in ONE artefact store.

I don't mean necessary abandoning the stores we have, it might make sense to have all psyclone output files in one artefact (and ditto for cofigurator etc), but there should be a place where all f90 and all c files can be found - so that means in some cases to add files to more than one artefact store.

There are two obvious ways of implementing this:

1. We just use some new custom artefact names, and no further code change. This is easy, what I don't like here is that we then need to import & use the name of these new stores
2. We supply custom functions (which will internally use a custom name, but that name does not need to be exposed).

For now I have implemented the 2.nd way, but if you prefer 1. or something entirely different, I won't mind. Example:

class ArtefactStore: 
...
    def add_fortran_build_files(self, files: Union[str, List[str], Set[str]]):
        self._add_files_to_artefact(self.FORTRAN_BUILD_FILES, files)

    def get_fortran_build_files(self):
        return self[self.FORTRAN_BUILD_FILES]

    def add_c_build_files(self, files: Union[str, List[str], Set[str]]):
        self._add_files_to_artefact(self.C_BUILD_FILES, files)

    def add_x90_build_files(self, files: Union[str, List[str], Set[str]]):
        self._add_files_to_artefact(self.X90_BUILD_FILES, files)
...

So, if for example the .f90 files are copied into the build tree, they will be added using add_fortran_build_files, and the analysis and compilation stage can just use get_fortran_build_files to get all files, as can custom steps (and they could e.g. remove and add files depending on what they do). No more need to pick files from three or four different places.

PSyclone would add its output files to the fortran build files as well.

Additionally, as indicated in the comment of the analysis step above - psyclone is very LFRic-specific, so the code will be much easier to understand.