ESPEI icon indicating copy to clipboard operation
ESPEI copied to clipboard

ENH: Allow multiple datasets directories to be specified in YAML input

Open bocklund opened this issue 4 years ago • 1 comments

Sometimes it is useful to load datasets from different filesystem locations, for example if one folder contains hand-curated data and another contains automatically generated data.

In code, it would be pretty simple to handle this. Instead of

from espei.datasets import load_datasets, recursive_glob
directory = '/path/to/directory/'
load_datasets(recursive_glob(directory))

we could do

from itertools import chain
from espei.datasets import load_datasets, recursive_glob
directories = ['/path/to/directory_1/', '/path/to/directory_2/']
load_datasets(chain(*map(recursive_glob, directories)))

bocklund avatar Oct 04 '21 19:10 bocklund

Alternatively:

from espei.datasets import load_datasets, recursive_glob
directories = ['/path/to/directory_1/', '/path/to/directory_2/']
globbed_files = sum(map(recursive_glob, directories), [])
load_datasets(globbed_files)

bocklund avatar Jan 06 '22 19:01 bocklund