[WIP] Simple timm splitter funcs

[tcapelle]

A simple method to split timm models. My main concer is that I am not createing the model_meta dict and instead at the moment when the model is created, I am constructing the splitter. I will train some models on monday. cc. @jp00

• requires timm from master

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[jph00]

Thanks for the great PR, @tcapelle ! I'll wait to hear back from @rwightman about a pypi release of the needed functionality before we merge this.

[tcapelle]

I am pasting this here. Have been running a bunch of sweeps comparing this timm split vs the default_split method here: https://wandb.ai/capecape/fine_tune_timm/sweeps .

(3 epoch finetune with 1/0 epoch fit)

A non-exhaustive exploration here shows that default_split works sometimes better for very short finetuning(3 epochs).

• resnet34

-regnetx_040

• convnext_tiny

This model is insane!! 🚀