ludwig icon indicating copy to clipboard operation
ludwig copied to clipboard

fix: Count full dataset in DatasetShard.__len__

Open jeffkinnison opened this issue 1 year ago • 1 comments

Running the new test with the previous code, DatasetShard.__len__ returned 256 rather than the full dataset size defined in the test. Summing the size of the batches returned by DatasetIterator.iter_batches returns the full dataset size.

jeffkinnison avatar May 21 '23 21:05 jeffkinnison

Unit Test Results

  6 files  ±0    6 suites  ±0   1h 19m 29s :stopwatch: + 8m 4s 33 tests ±0  29 :heavy_check_mark: ±0    4 :zzz: ±0  0 :x: ±0  99 runs  ±0  87 :heavy_check_mark: ±0  12 :zzz: ±0  0 :x: ±0 

Results for commit 8ff5f4d3. ± Comparison against base commit cb37535c.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar May 21 '23 23:05 github-actions[bot]