nowcasting_dataset icon indicating copy to clipboard operation
nowcasting_dataset copied to clipboard

Experiment with loading entire batches at once

Open JackKelly opened this issue 4 years ago • 0 comments

Detailed Description

The code currently loads invidual training examples, one-at-a-time, and then joins them into a batch.

But it's possible that it might be faster to execute, and the code might be simpler, if we load entire batches at once. (For example, I think Zarr might be able to concurrently load all the examples).

I have a hunch that the "loading single examples at a time" thing might not be necessary once we do #202

JackKelly avatar Oct 08 '21 15:10 JackKelly