Percy Li
Percy Li
You might find it helpful to check out the vLLM or Ollama community for this kind of assistance. I'd suggest giving Ollama a try. It works really well with dspy,...
Hello, everyone. I've gone through all the posts here and it seems that no one else has encountered this particular issue. I'm wondering if perhaps I'm not using it correctly....
The only modification needed is to change the size of token embeddings. Just insert this line: shared.model.resize_token_embeddings(len(shared. Tokenizer)) Do NOT replace the original code with: shared. Model = PeftModel.from_pretrained(shared.model, Path(f"{shared.args.lora_dir}/{lora_name}"),...