ragflow icon indicating copy to clipboard operation
ragflow copied to clipboard

Add configuration to choose default llm models

Open Omarlsg98 opened this issue 10 months ago • 0 comments

What problem does this PR solve?

This pull request includes changes to the api/settings.py and docker/service_conf.yaml.template files to add support for default models in the LLM configuration (specially for LIGHTEN builds). The most important changes include adding default model configurations and updating the initialization settings to use these defaults.

For example: With this configuration Bedrock will be enable by default with claude and titan embeddings.

user_default_llm:
  factory: 'Bedrock'
  api_key: '{}' 
  base_url: ''
  default_models:
    chat_model: 'anthropic.claude-3-5-sonnet-20240620-v1:0'
    embedding_model: 'amazon.titan-embed-text-v2:0'
    rerank_model: ''
    asr_model: ''
    image2text_model: ''

Type of change

  • [ ] Bug Fix (non-breaking change which fixes an issue)
  • [X] New Feature (non-breaking change which adds functionality)
  • [ ] Documentation Update
  • [ ] Refactoring
  • [ ] Performance Improvement
  • [ ] Other (please describe):

Omarlsg98 avatar Feb 21 '25 19:02 Omarlsg98