Stealthwriter
Stealthwriter
Hi, can you please add documentation to self-host on runpod? it offers GPU prices cheaper than aws.
Hi, Is there a way to change the frequency_penality or logit bias when sending a completion request?
### System Info Hi, on inference endpoint in huggingface the TGI for classifiers is working but here it doesn't, Deberta v3 classifier is not supported? ### Information - [ ]...
### 🐛 Bug ARNING] using untested triton version (2.3.1), only 1.0.0 is known to be compatible [WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.3...
### Checklist - [X] 1. I have searched related issues but cannot get the expected help. - [X] 2. The bug has not been fixed in the latest version. -...
### Please check that this issue hasn't been reported before. - [X] I searched previous [Bug Reports](https://github.com/axolotl-ai-cloud/axolotl/labels/bug) didn't find any similar reports. ### Expected Behavior it should train ### Current...
If I send too many requests, like 2000 per second, the gateway can't handle that. Also, I have 9 services running. We need to either run multiple gateways or at...
### System Info response_format doesn't work with open ai endpoint, please add it ### Information - [x] Docker - [ ] The CLI directly ### Tasks - [x] An officially...
### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. -...
I'm getting this error while I start the server, what could be the issue? and how to fix it please? `{"level":"error","time":"2025-09-17T07:50:23.364Z","pid":1,"hostname":"b2a0c7bbbffa","err":{"type":"Error","message":"Migration failed. Reason: An error occurred running 'prices'. Rolled back...