Julia Merz

Default String for ConfiguredSamplers

1

https://github.com/rustformers/llm/blob/18b2a7d37e56220487e851a45badc46bf9dcb9d3/crates/llm-base/src/samplers.rs#L95 We have a default for ConfiguredSamplers, as well as a FromStr implementation for users to give in a sampler string. However, at the moment we do not have that...

Proper Rewind+Refeed when stop token is detected.

1

[See this segment of code](https://github.com/rustformers/llm/blob/bca64b00c936d649bb95c58993eff0bb43c4f541/crates/llm-base/src/inference_session.rs#L956) ``` rust /// An [InferenceResponse] callback that will halt inference when a `stop_sequence` is generated. /// This callback is used in [InferenceSession::infer] in chat_mode. pub...