SwayStar123

try fsdp or deepspeed

The tutorial_test_dataset works fine and shows both images as having the same shape

I think I rounded all my images to nearest 64 on both axis and that solved it

@gg-yunse I meant the nearest multiple of 64, like 128, 192, 256, 320, and so on

use fsdp or deepspeed training strategy

Just change the filename lol

Would allowing users to extend this to their projects errors be possible aswell?

I think worths talking discussing if we should have two libraries in the std lib, one with the same api as before (single token), and one with the above multi-token...

@sudhackar Hi, feel free to make a PR to fix this issue, after you add the implementation and the tests you can request a review from me

There is no u8 because you dont need it for u8, to convert a u8 to an array of byte(s) you just put it in an array, no additional logic...