lm-evaluation-harness
lm-evaluation-harness copied to clipboard
Add ability to specify custom local completions headers
trafficstars
I started thinking about this after testing a model hosted via Kserve inference service, which supports OpenAI APIs for certain serving runtimes. My setup uses an ingress gateway (no DNS) and, as a result, I need to supply the "Host" header in addition to the base URL. I'm not sure it's worth subclassing the LocalCompletions class in this case.
I would appreciate the community's thoughts on having the ability to specify custom headers on the command line. There may be certain implications of this that I haven't considered :)