jetstream-pytorch icon indicating copy to clipboard operation
jetstream-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon google

[Feature Request] Per request sampling params

Open qihqi opened this issue 5 months ago • 3 comments

Currently sampling params such as temperature are set as commandline flags in when the server starts.

It would be nice for each request to pass in the sampling params instead.

qihqi avatar Sep 24 '24 17:09 qihqi