Omer Shamash

Basically classes such as `StreamingResponse` in `starlette` cannot work unless specified directly in the response-type due to this serialization concept (which they always fallback to use JSON) There is the...

[Question]: Running LLMLingua with GGUF models

1

### Describe the issue Hi, trying to make it run properly with GGUF models (i.e. CPU only) due to RAM restriction, Trying to use it as ``` compressor = PromptCompressor(...