lbann icon indicating copy to clipboard operation
lbann copied to clipboard

Data_* Cleanup

Open bvanessen opened this issue 1 year ago • 0 comments

  • [ ] Input layer output size is grabbed directly from data coordinator
  • [ ] remove access to the global mini-batch size - all mini-batch buffer sizes are derived
  • [ ] identify, correct, or deprecate JAG data reader issues that are encumbering process
  • [ ] multi-key-value input data reader
  • [ ] remove data_* from everywhere
  • [ ] callbacks???
  • [ ] input layer no longer modifies data reader / coordinator state. Move to outside of model
  • [ ] Get sample indices per mb
  • [ ] get num iterations per epoch - move to SGB
  • [ ] The location of shuffle_required should be moved out of data reader metadata

bvanessen avatar Oct 02 '23 19:10 bvanessen