Joe Cummings
Joe Cummings
Unfortunately, it looks like right now in [`get_optimizer_state_dict`](https://github.com/pytorch/pytorch/blob/134179474539648ba7dee1317959529fbd0e7f89/torch/distributed/checkpoint/state_dict.py#L781), it expects at most one `nn.Module` for the model param. As such, the simplest thing to do (albeit not the most efficient...
This seems like a super cool usecase @almugabo ! Lemme ask a couple follow up questions: - Can you provide an example of a sample from the dataset? I'm just...
ADDED!
CC @winglian for his thoughts on caching the packed datasets. I believe axolotl already does something like this?
> @RdoubleA If our plan is to move to online packing anyway, does this make sense as a direction to go in? This wouldn't be fully online though, right? And...
Ah apologies @stas00 ! We have a note [here](https://github.com/pytorch/torchtune#installation-%EF%B8%8F) that says we only test on the latest PyTorch stable and PyTorch nightlies, but we should definitely make that more clear....
@stas @janeyx99 Can this be closed with #2627 and #2626 ?
> Understood! > > Would it too difficult to do a runtime check for pytorch version in `__init__.py` and tell the user if there is a mismatch and what exact...
I can definitely understand where you're coming from and for the most part, when we detect that a certain feature is only available after a specific version, we do attempt...
No, thank you for your feedback! > If older versions are in "may or may not be supported" it's an ambiguous state which would result in users leaving because they...