Daniel Han comments

Results 983 comments of


                                            Daniel Han

Merging to 16bit for vLLM produces lower performance

I'm working on a new method which will make this better!

Merging to 16bit for vLLM produces lower performance

@brando90 The issue is sometimes merging to 16bit from upcasting can cause accuracy issues

convert-hf-to-gguf.py should have underscores.

Oh yes!! Actually would you be interested in opening a PR and editing the Readme file, and then I'll copy paste your edits to the Wiki :)

On train_on_responses_only

TRL's Data Collator does not work on multiple conversations, and only works on 1 conversation.

only instruct model can use Llama-3 prompt format

Oh it's because the base model has untrained tokens - see https://unsloth.ai/blog/phi3 (Phi-3 blog has Llama-3 fixes). We identified this issue about using the Llama-3 chat template for the base...

Request: Flux (Diffusion transformer)

Diffusion / Llava type models are next on our roadmap!

Request: Flux (Diffusion transformer)

@al-swaiti Yep working on them!

llama3.1-8b Guff Conversion Failure

Oh did you add new tokens?

Can we fine tune a fine tuned model which was not trained by unsloth using unsloth?

Yes! Simply change the model name - we'll error out if the model does not work

Gemma 2's 2B LoRA adapter merge is not working

Ok that's weird its not merging correctly? I'll check