Ross Wightman

@adamjstewart if you need any points re 'modern timm' let me know, for the most part, especially if it was mostly just models being used and not importing less used...

appreciate the submission, but added complexity is not worth it imho, torchcompile also changes the equation quite a bit.

torch.manual_seed(seed) torch.cuda.manual_seed(seed) torch.cuda.manual_seed_all(seed) that's redundant, if you are using typical distributed training with one GPU per process, just one call to torch.manual_seed() is needed, cuda is called if it's there...

See also, https://github.com/huggingface/pytorch-image-models/pull/853 I don't believe full determinism is possible in all cases for all models due to some ops just not having support (at least last time it was...

@ekurtulus that sounds interesing, can it be implement similar to augmix + jsd loss where most of the detail wrt to the splits of data, etc is in the dataset...

@mjamroz does this cover the case of a DDP wrapped torchcompile model? I don't know if i've actually checked that ... is it _orig_mod.module. or just _orig_mod?

@LuoXin-s The OpenAI CLIP ResNets have an attention pooling layer that makes the models quite a bit larger than same depth ResNet (significant FLOPs and params in the layer), and...

It also uses a non-trivial head, needs features from more than one spot, not too crazy, but needs design for a custom head + backbone that I've thought of, but...

@a-r-r-o-w so yeah, there are a LOT of missing type hints and docstrings, some more recent models have some, most older ones do not. I'm open to PRs, as it'll...

@yazdanimehdi finally got around to picking up a 4090. It's nice and decent boost when using torchcompile. I tried fiddling with Transformer Engine and FP8 autocast and it wasn't very...