Benjamin Bossan

I agree, this requires either digging deeper into how `evaluate` calculates the metrics or just rolling a custom metric (or maybe use [sklearn](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.jaccard_score.html)?). We can keep this issue open until...

@sambhavnoobcoder Are you still working on this?

Thanks @sambhavnoobcoder, the PR looks good from my side. @githubnemo could you please also do a review :pray:?

Thanks for the suggestion. Yes, eventually we should move to `pyproject.toml` and remove `setup.py` and `setup.cfg`. As we're not doing anything fancy there, it should be possible without losing any...

Thanks for reporting this error. We cannot change the argument just like that, as this will lead to failure in loading other models. Instead, let's try to debug why Flux...

> Should we add a special flag for peft Flux compatible? Before we do that, we need to first understand why this adapter causes the issue, while others work. Then...

I have a bit of time to investigate the issue this week. Do you know of a publicly available LoRA Flux adapter that causes the issue you described (only safetensors)?...

_not stale_

@tomguluson92 Did you see [my last question](https://github.com/huggingface/peft/pull/2240#issuecomment-2511561704)?

@nsbg A first step would be to find a way to replicate the error. If you're interested in working on this, I'd be happy to assist.