lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

Add ability to specify custom local completions headers

Open kiersten-stokes opened this issue 8 months ago • 0 comments
trafficstars

I started thinking about this after testing a model hosted via Kserve inference service, which supports OpenAI APIs for certain serving runtimes. My setup uses an ingress gateway (no DNS) and, as a result, I need to supply the "Host" header in addition to the base URL. I'm not sure it's worth subclassing the LocalCompletions class in this case.

I would appreciate the community's thoughts on having the ability to specify custom headers on the command line. There may be certain implications of this that I haven't considered :)

kiersten-stokes avatar Mar 10 '25 16:03 kiersten-stokes