Scott Hoang

this repo is good ONLY for INFERENCE with PROVIDED WEIGHTS.

8

like the title said, this implementation suffers serious issue when trained from scratch with mAP stalling at around 20%. If you are planning to train this on custom data, I...

TypeError: EViT.__init__() got an unexpected keyword argument 'default_cfg'

3

When running run_code.sh I get this error. Can you help?

Request support for Eleuther generative task

1

Hi all, Thank you for the excellent work you guys have been doing thus far. I request that we add support to Eleuther's generative tasks. We are not encoding batch...

clarification on how to interpret kernel size for conv1d

1

Can we interpret the convolution kernel size as the context length? Would increasing kernel size allows longer range context?

Changing expansion_factor resulted in 'c10::DistBackendError

1

Are there any dimensional constraints when using expansion_factor? i.e., the dimension must be multiple of something? Thanks!

To accelerate training, it is common practice to pack many text sequences into the same sentence. Self-attention avoids sample contamination with causal mask/sequence IDs. I do not see a similar...

safe_torch_load failed when resume from checkpoint

6

Location: torchtune/utils/_checkpointing/_checkpointer.py line: 438 Error: _pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only...

[feature request] Saving / Loading packed dataset

8

Hi team, Can we save and load a packed dataset? I have a used case where I must train multiple models on the same packed dataset with identical sequence lengths....