Benjamin Bossan

Thanks for elaborating Thiago. I agree that there can be scenarios where a flag to turn off accelerate would be useful. My feeling is, however, that in the vast majority...

This should not happen, thanks for reporting this issue. > but noticed that the accuracy drops by ~4.5% when doing so Is this train or validation/test accuracy? Are those absolute...

Thanks for the details. Unfortunately, this is not enough for us to pinpoint the exact issue could be. I have 2 suggestions: - Take the usual steps to mitigate NAN...

@skylooop Without the full code and data, we cannot start debugging this issue. If you cannot share, can you identify the exact version of PEFT at which training starts to...

Thanks for the pointer. I tried replicating your issue by using the [fine-tuning script](https://github.com/microsoft/TransformerCompression/tree/main?tab=readme-ov-file#recovery-fine-tuning) but ran out of memory. I tried lowering some hyper-params but still no luck, and when...

> I found the same problem when trying to peft fine-tune CodeLLama-7B (using LlamaForSequenceClassification), the Loss is always 0 during the fine-tuning. Is that also with the MS compression library?...

Thanks for providing more details. I did a quick check on the diff between v0.6.0 and v0.7.0 but at first glance, nothing came up that could explain the difference that...

@sssszh Thanks for providing the script. I didn't have access to your data, so I requested it. Would this also work with another dataset, like one of the datasets on...

> This dataset is also available on huggingface: `claudios/code_x_glue_devign`, but you may need to modify the code in the Dataset section, because the format of the data is not the...

Thanks for providing access, I could finally run the example. First of all, I had to make a few small modifications to the script because of OOM errors, but I...