nowcasting_dataset
nowcasting_dataset copied to clipboard
enable nowcasting dataloader to append to the CSV files to create more batches
At the moment, the workflow is:
- Specify the number of batches for each split in the config YAML
- Run
prepare_ml_data.py
. Which first creates CSV files specifying the locations of each example. - Once the CSV files exist, it'd then possible to re-create batches for each modality. But it's not possible to increase the number of batches requested in the config YAML.
There should be a command line-option like --append_more_examples_to_csv_files
which, as the name suggests, increases the number of examples specified in the CSV files, and then creates those new batches for each modality.