Steffen Röcker
Steffen Röcker
I tried to do this here: https://github.com/sroecker/llama.cpp/tree/add_mlp_bias Just adding bias to FFN_GATE, FFN_DOWN and FFN_UP. The tensor shapes seem to be correct but the model outputs gibberish. `./main -m ~/Downloads/granite-3b-code-base.Q8_0.gguf...
Ah, not yet. Thanks! I guess then we need to define an additional ARCH (or save the mlp_bias boolean in the GGUF) and implement it like with MPT https://github.com/ggerganov/llama.cpp/blob/7e0b6a7b3ba94ff624dc27c1e0e735fded8819b8/llama.cpp#L5287 Mayank...
> > (or save the mlp_bias boolean in the GGUF > > Does exist a way to add mlp_bias to already made gguf? I ask about that because you mentioned...
So I've adapted build_llama to include the MLP biases as well. I've added a few FIXMEs to my branch to indicate places that might need to be adapted for the...
> AFAIK, we have been stuck on the issue of repeating text output. It appears that the tokenizer is the culprit, but it does seem to be in order, correct...
> the lab version is a different model not to be confused with this one I'm aware of that, it did work out of the box with LLM_ARCH_LLAMA settings though...
> Hmm, a quick question: are we tying the word embeddings and output logits matrix? llama doesn't do that and granite has tied embeddings. maybe thats the issue? I don't...
> Hmm, ok so there are these differences between llama and granite: > > 1. attention has bias (llama doesn't) > 2. mlp has bias (llama doesn't) > 3. tied...
Hi, this is great to hear! You could configure nginx as a proxy for the streamlit container as described here: https://discuss.streamlit.io/t/deploy-streamlit-with-nginx-docker/52907/2 Let me know if that works. Nabelou Ouologuem ***@***.***>...
> same issue . i think if [`context_scoring_parser = RagasoutputParser(pydantic_object=ContextScoring)` in `output_parser.py` ](https://github.com/explodinggradients/ragas/blob/9bc6e6fd44180e658751e15da3a3829c957ee853/src/ragas/llms/output_parser.py#L61) return None , means llm cannot gen result by right prompt , it will cause this exception....