pytorch-image-models icon indicating copy to clipboard operation
pytorch-image-models copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Resuming compiled model checkpoint

Open mjamroz opened this issue 1 year ago • 2 comments

Sometimes im getting Missing key(s) in state_dict: "stem.conv1.c.weight", [...] Unexpected key(s) in state_dict: "_orig_mod.stem.conv1.c.weight", [...] while trying to resume training from checkpoint of compiled (--torchcompile=inductor) model. This PR solves the issue by stripping _orig_mod. prefix before loading weights.

mjamroz avatar Oct 23 '23 07:10 mjamroz

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

HuggingFaceDocBuilderDev avatar Oct 23 '23 07:10 HuggingFaceDocBuilderDev

@mjamroz does this cover the case of a DDP wrapped torchcompile model? I don't know if i've actually checked that ... is it _orig_mod.module. or just _orig_mod?

rwightman avatar Nov 02 '23 19:11 rwightman