Ismail
Ismail
Hello @tianyu-l, Thank you very much! I'd be honoured to work with you on this task! Yes, discussing the design will be helpful and could prevent extra work. I was...
Dear all, in-training validation and evalution is an important feature for many users including myself. So far I have been exploring/trying torchtune and its Eleuther eval harness builtin evaluation as...
@tianyu-l Absolutely!
@tianyu-l Thanks!
Minimal example to reproduce this behavior: ```python import torch import kan torch.save(kan.KAN([5,5]), 'test.pt') ``` This last gives the error: ```AttributeError: Can't pickle local object 'Symbolic_KANLayer.__init__..zero_fun'``` A workaround using `dill` package:...
I noticed a spike in train loss at step 150 (warmup ends at step 200) for the current default config for llama3.1 8b using 8192 sequence length. https://github.com/pytorch/torchtitan/blob/781ec0d0187e69935449b5f98b8f60af0dc5091f/torchtitan/models/llama3/train_configs/llama3_8b.toml This spike...