camel
camel copied to clipboard
feat: Implement structured output by using outlines
This PR introduces structured output capabilities for both open-source and closed-source language models (LLMs) in our system:
-
Open-source LLMs:
- Implemented a schema model to integrate open-source, self-hostable LLMs supported by Outlines
- This enables structured output for open-source models
- Currently supports platforms: Transformers, LlamaCpp, and vLLM
-
Closed-source LLMs with function calling:
- Utilized Pydantic to pass expected output structures
- This approach enables structured output for closed-source models that support function calling
Key changes:
- Added SchemaModel class for open-source LLM integration
- Implemented platform-specific adapters for Transformers, LlamaCpp, and vLLM
- Created PydanticOutputParser for closed-source LLMs with function calling support
Testing:
- Added unit tests for SchemaModel and PydanticOutputParser
Next steps:
- Expand support for additional open-source LLM platforms
- For the closed llms, we will optimize the issue of multi-functions calling at the same time.
- Better ci/cd for the self-hosted models