[Feature Request]: Token compression using GPT-3.5-turbo

[ohdearquant]

Is your feature request related to a problem? Please describe.

local models are too slow in compressing tokens, cannot fulfill needs of bigger dataset

Describe the solution you'd like

instead of only relying on hugging face, provide API backed models for compression

Additional context

No response

[iofu728]

Hi @ohdearquant, thank you for your suggestion. We have plans to support the API-based model as a compressor.

related issue #44.

[younes-io]

@iofu728, is there any timeline for when this will land?

[iofu728]

Hi @younes-io, there are still some blocking issues that need to be resolved. Once they are addressed, we will promptly support the corresponding feature.