anything-llm
anything-llm copied to clipboard
[FEAT]: HF Serverless Inference API
What would you like to see?
The hugging face widget only allows for dedicated inference API endpoints. There are several free-to-trial serverless API endpoints that would not have the '.cloud' suffix.
such as the new https://huggingface.co/CohereForAI/c4ai-command-r-v01
https://huggingface.co/docs/api-inference/index
I would like to see a second dialogue box for serverless APIs or maybe a different widget or maybe the same widget but include both dedicated and serverless.