outlines icon indicating copy to clipboard operation
outlines copied to clipboard

Clean Up Dead `outlines.integrations` Code

Open lapp0 opened this issue 1 year ago • 1 comments

What behavior of the library made you think about the improvement?

With all models other than exllamav2 having a single integration implementation via outlines.generate.api.SequenceGeneratorAdapter and outlines.processors, we don't need outlines.integrations.

  • outlines.integrations.transformers has RegexPrefixAllowedTokens and JSONPrefixAllowedTokens which are now handled by outlines.processors
  • outlines.integrations.llamacpp has logits processors which are redundant with outlines.processors
  • outlines.integrations.vllm has logits processors which are redundant with outlines.processors
  • outlines.integrations.utils has adapt_tokenizer, useful for FSM construction, and convert_json_schema_to_str, used for the three model integrations listed above

After the vLLM integration, we no longer need outlines.integrations

How would you like it to behave?

Get rid of outlines/integrations/ and tests/generate/test_integration_*.py

  • Remove outlines.integrations.transformers, outlines.integrations.llamacpp, and outlines.integrations.vllm
  • Remove outlines.integrations.utils, but move def adapt_tokenizer(...) to outlines.models.tokenizer

lapp0 avatar Jul 20 '24 07:07 lapp0

We should do this once #1010 has been merged and then release 0.1.0

rlouf avatar Jul 20 '24 16:07 rlouf