l3utterfly
l3utterfly
Digging deeper into the code, this is where it went wrong: `/axolotl/src/axolotl/prompters.py:325` The `Conversation` object from fast chat has hard coded `sep2=""` Temporary workaround for phi, need to add the...
This line also needs to be added: `self._conversation.sep2 = ''` in `/axolotl/src/axolotl/prompters.py:286`
Here is the config: ``` base_model: /home/layla/src/text-generation-webui/models/Mistral-7B-v0.1 base_model_config: /home/layla/src/text-generation-webui/models/Mistral-7B-v0.1 model_type: MistralForCausalLM tokenizer_type: LlamaTokenizer is_mistral_derived_model: true load_in_8bit: false load_in_4bit: false strict: false datasets: - path: /data.jsonl ds_type: json # see other...
@belladoreai thank you so much! I'll keep an eye on your original PR in oobabooga to see if the maintainer has any reviews/suggestions. If all's good, I'll port it to...
> > I'd really love to see this merged, is there anything that needs to be done before that happens? > > Also needs unit tests [#6445 (comment)](https://github.com/ggerganov/llama.cpp/pull/6445#issuecomment-2040373416) I've synced...
> @l3utterfly Do you need support with any of these last parts, whether testing or what @p-e-w mentioned with CLI parameters and such to finish this PR out? My C++...
> Do we ? Does it need to be implemented here https://github.com/ggerganov/llama.cpp/blob/master/common/sampling.cpp#L245 ? > > `llama_sample_dry` to be referenced somewhere The sampler is called here: https://github.com/ggerganov/llama.cpp/pull/6839/files#diff-50fcac93c9d3fb912c9f30db3657a5416d65238b22683b704cfe20071123a867R432 It goes before your...
> While I have used git a good deal for personal projects, I am a complete noob with github so if I screwed something up or am not following proper...
So I need to use pytorch to save the state dict file? I tried that with a Llama3 fine tune and then tried to compile it for XNNPACK, I got...
@iseeyuan sorry, I'm a little new to torch tune, following the documentation here: https://pytorch.org/torchtune/stable/deep_dives/checkpointer.html#understand-checkpointer 1. I am loading my safetensor file first: ``` AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16, device_map="cpu", ) ``` 2....