higress
higress copied to clipboard
Support modifying model name based on model mapping when falling back to the target service
When using the fallback capability, it is similar to the situation in #1901 , where model mapping cannot be set.
The configuration method also needs to be modified in a similar way.
Can be changed to the following configuration:
apiVersion: extensions.higress.io/v1alpha1
kind: WasmPlugin
metadata:
name: model-mapper.internal
namespace: higress-system
spec:
defaultConfigDisable: true
failStrategy: FAIL_OPEN
matchRules:
- config:
modelMapping:
'gpt-4-*': "qwen-max"
'gpt-4o': "qwen-vl-plus"
'*': "qwen-turbo"
configDisable: false
# This configuration takes effect when the following ingress and service match simultaneously.
ingress:
- ai-route-dashscope.internal
service:
- llm-dashscope.internal.dns
- config:
modelMapping:
'gpt-4-*': "qwen-plus"
'*': "qwen-long"
configDisable: false
# This configuration takes effect when the following ingress and service match simultaneously.
ingress:
# When fallback is triggered, it will go through this ingress.
- ai-route-dashscope.fallback.internal
service:
- llm-dashscope.internal.dns
phase: AUTHN
priority: 800
url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/model-mapper:1.0.0