Evan Griffiths

icon company Grantify icon location Glasgow / Remote icon location LLM Engineer at grantify.io. MEng in information engineering from Cambridge

Results 2 issues of Evan Griffiths

Use huggingface tokenizer chat template for inference

1 comment

In the llm model go code (e.g. [here](https://github.com/helixml/helix/blob/main/api/pkg/model/mistral7b.go#L46)) we build up a prompt that is a formatted string based on the chat template associated with the model. We could instead...

Add token count to return of FactCheck().check_response

1 comment

It would be nice to be able to track tokens used by a fact check call using the python api, e.g. something like: ``` from factcheck import FactCheck results =...

enhancement