50h100a

Includes typehint corrections and the discussed changes to prompt logprob sampler values.

Reduce peak memory for prompt_logprobs requests

1

First order of business, make prompt_logprobs "compatible" with prefix caching. It can't take advantage of the caching, but at least it will run. Second order of business, reduce the peak...