[AI] feat: Implement LLM proxy filter with retry and fallback mechanisms

[Alanxtl]

New LLM Proxy Filter Implementation:

• Implemented the pkg/filter/llm/proxy/filter.go file, which includes the LLM Proxy filter's logic for handling HTTP requests, retries, and fallbacks.

Usage

static_resources:
  listeners:
    - name: "llm_proxy"
      protocol_type: "HTTP"
      address:
        socket_address:
          address: "0.0.0.0"
          port: 8888
      filter_chains:
        filters:
          - name: dgp.filter.httpconnectionmanager
            config:
              route_config:
                routes:
                  - match:
                      prefix: "/chat/completions"
                    route:
                      cluster: "chat"
                      cluster_not_found_response_code: 505
              http_filters:
                - name: dgp.filter.llm.proxy
                  config:
                    maxIdleConns: 100
                    maxIdleConnsPerHost: 100
                    maxConnsPerHost: 100
                    scheme: "https"
                # - name: dgp.filter.llm.tokenizer
      config:
        idle_timeout: 5s
        read_timeout: 5s
        write_timeout: 5s
  clusters:
    - name: "chat"
      lb_policy: "lb"
      endpoints:
        - id: 1
          llm_meta:
            provider: "deepseek"
            fallback: true
            retry_times: 5
        - id: 2
          llm_meta:
            provider: "deepseek"
            retry_times: 3
            fallback: true
      health_checks:
        - protocol: "https"
          timeout: 1s
          interval: 5s
          healthy_threshold: 4
          unhealthy_threshold: 4
  shutdown_config:
    timeout: "60s"
    step_timeout: "10s"
    reject_policy: "immediacy"

[sonarqubecloud[bot]]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
25.9% Duplication on New Code

See analysis details on SonarQube Cloud