notebooks icon indicating copy to clipboard operation
notebooks copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

instruction finetuning IDEFICS

Open schwarzwalder93 opened this issue 1 year ago • 0 comments
trafficstars

Hi Team,

I would like to instruct fine tune IDEFICS on custom dataset. Could you provide the training script used to produce idefics-9b-instruct ?

I suppose changes will be need to following snippets:

labels should correspond to responses/answers to the input instructions: https://github.com/huggingface/notebooks/blob/1dda4aa021512743176cc69cb73df341a969a99a/examples/idefics/idefics_zero3_finetuning/idefics_zero3_finetuning.py#L71

Modifying the logic before loss computation: https://github.com/huggingface/transformers/blob/bd50402b56980ff17e957342ef69bd9b0dd45a7b/src/transformers/models/idefics/modeling_idefics.py#L1508C1-L1523C97

schwarzwalder93 avatar Nov 29 '23 05:11 schwarzwalder93