Lukas Kreussel
Lukas Kreussel
This should be solved with the last GPT-2 changes.
Couldn't we use huggingfaces tokenizer? Then we would have parity with nearly every implementation out there 🤔
Redpajama is a GPTNeoX based model, which means you should be able to run it via the gptneox example.
> That's the claim I want to explore. In my tests, some model performed way worse and it's not 100% clear why. I would greatly appreciate any benchmarks/test of the...
You can resolve this by using the `network` method and only installing the packages you actually need. This also makes the installation a lot faster. (For all available packages see...
I guess this could also be realated to phi-3 producing gibberish output on long input sequences?
TEI defined [their triggers](https://github.com/huggingface/text-embeddings-inference/blob/main/.github/workflows/build_89.yaml#L5C1-L9C14) as follows: ``` on: workflow_dispatch: push: branches: - 'main' tags: - 'v*' ``` We only need the `tags` part of it. The `release` trigger you are...
Probably, easiest way to find out is to just try it. Github actions can be very finicky sometimes 😅
@lucasavila00 I think Laurent added the batched kernels today: https://github.com/huggingface/candle/pull/2060
In my oppinion 3 would also be the best option, especially if it would be possible to define the adapters to use per request to the server. (This could be...