higress icon indicating copy to clipboard operation
higress copied to clipboard
verified publisher icon alibaba

ai统计插件bug

Open ilovedumplings opened this issue 9 months ago • 20 comments

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

  • [ ] I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

当开启ai统计(最新版本,sha256 c1e2453c02e823143a9d2fda047949f0ca2920ea2e268b10634e3ab3003d6732)插件的时候,ai配额会失效,失效日志如下: 日志里没有输出关于update相关的信息

Image

当ai统计插件更换为老版本的时候(sha256 3943fb35dddecea22d9ffc29f5baba7f42bdf4d850467c782cf6a81f4ae1df56) 会打印update日志

Image

Ⅱ. Describe what happened

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Ⅴ. Anything else we need to know?

@CH3CHO @johnlanni 请麻烦看下这个问题,应该是个bug

Ⅵ. Environment:

  • Higress version:
  • OS :
  • Others:

ilovedumplings avatar Mar 10 '25 09:03 ilovedumplings

@rinfx

ilovedumplings avatar Mar 10 '25 09:03 ilovedumplings

quota 插件是不是用的老版本？quota插件老版本依赖统计插件将token信息写入到filter_state中，新的quota插件解除了这个限制

rinfx avatar Mar 17 '25 09:03 rinfx

quota 插件是不是用的老版本？quota插件老版本依赖统计插件将token信息写入到filter_state中，新的quota插件解除了这个限制

Image 使用的是latest

ilovedumplings avatar Mar 19 '25 02:03 ilovedumplings

quota

那是quota插件的问题还是统计插件的问题??

ilovedumplings avatar Mar 19 '25 02:03 ilovedumplings

apiVersion: extensions.higress.io/v1alpha1 kind: WasmPlugin metadata: annotations: higress.io/wasm-plugin-description: Implement quota-based rate limiting according to assigned fixed quotas, while also supporting quota management capabilities, including querying, refreshing, and adjusting quotas. higress.io/wasm-plugin-icon: https://img.alicdn.com/imgextra/i1/O1CN018iKKih1iVx287RltL_!!6000000004419-2-tps-42-42.png higress.io/wasm-plugin-title: AI Quota creationTimestamp: "2024-11-24T07:08:51Z" generation: 117 labels: higress.io/resource-definer: higress higress.io/wasm-plugin-built-in: "true" higress.io/wasm-plugin-category: ai higress.io/wasm-plugin-name: ai-quota higress.io/wasm-plugin-version: 1.0.0 name: ai-quota-1.0.0 namespace: higress-system resourceVersion: "42954554" selfLink: /apis/extensions.higress.io/v1alpha1/namespaces/higress-system/wasmplugins/ai-quota-1.0.0 uid: f04b3239-831c-4e87-9074-e6c0f0a9a3b6 spec: defaultConfig: admin_consumer: consumer3 admin_path: /quota redis: service_name: redis.static service_port: 80 timeout: 2000 redis_key_prefix: 'chat_quota:' defaultConfigDisable: true matchRules:

  • config: admin_consumer: 98aaa0ab-fd20-4164-8c59-bda9970622d0 admin_path: /819c6419-4ee9-4ae9-a827-b44e60027148 redis: password: 12312344 service_name: redis.static service_port: 80 timeout: 2000 redis_key_prefix: 8a016c3f-9486-4d92-8660-cc51fad3f892 configDisable: false ingress:
    • qwen-public phase: UNSPECIFIED_PHASE priority: 750 url: oci://higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-quota:latest

apiVersion: extensions.higress.io/v1alpha1 kind: WasmPlugin metadata: annotations: higress.io/wasm-plugin-description: Provides statistics of token usage, including logs, monitoring, and alerts. higress.io/wasm-plugin-icon: https://img.alicdn.com/imgextra/i1/O1CN018iKKih1iVx287RltL_!!6000000004419-2-tps-42-42.png higress.io/wasm-plugin-title: AI Statistics creationTimestamp: "2025-03-17T10:28:01Z" generation: 14 labels: higress.io/resource-definer: higress higress.io/wasm-plugin-built-in: "true" higress.io/wasm-plugin-category: ai higress.io/wasm-plugin-name: ai-statistics higress.io/wasm-plugin-version: 1.0.0 name: ai-statistics-1.0.0 namespace: higress-system resourceVersion: "43227446" selfLink: /apis/extensions.higress.io/v1alpha1/namespaces/higress-system/wasmplugins/ai-statistics-1.0.0 uid: 7fcc2aee-fa0b-413d-a3a5-36e7ab97279b spec: matchRules:

  • config: enabled: true configDisable: false ingress:
    • qwen-public phase: UNSPECIFIED_PHASE priority: 200 url: oci://harbor.cloud.com/plugins/ai-statistics:latest-39

当ai-statistics插件的的sha256为c1e2453c02e823143a9d2fda047949f0ca2920ea2e268b10634e3ab3003d6732时(最新的latest),不会消耗token 当插件的sha256为3943fb35dddecea22d9ffc29f5baba7f42bdf4d850467c782cf6a81f4ae1df56是,会消耗token

ilovedumplings avatar Mar 19 '25 06:03 ilovedumplings

判断是否消耗token的方式就是看ai-quota日志是否打印update consumer日志即可

ilovedumplings avatar Mar 19 '25 06:03 ilovedumplings

cc @rinfx 近期修复一下吧

johnlanni avatar Mar 19 '25 10:03 johnlanni

cc @rinfx 近期修复一下吧

老哥这个有时间点没?我们周五上线了QAQ

ilovedumplings avatar Mar 19 '25 11:03 ilovedumplings

@ilovedumplings 我重新推了下quota的镜像，你再试一下

rinfx avatar Mar 21 '25 01:03 rinfx

quota

这个是修复ai-quota插件,不是修复ai统计插件么? @rinfx

ilovedumplings avatar Mar 24 '25 01:03 ilovedumplings

Image

Image

Image 测试了下,还有问题的..没有打印update consumer 配额没有更新 @rinfx

ilovedumplings avatar Mar 31 '25 08:03 ilovedumplings

日志里面respQuery字符串可以base64 --decode一下看看内容

rinfx avatar Mar 31 '25 08:03 rinfx

日志里面respQuery字符串可以base64 --decode一下看看内容 @rinfx

decode出来就是剩余的token数量

Image

ilovedumplings avatar Mar 31 '25 08:03 ilovedumplings

JDYNCjUwMDAwMA0K

ilovedumplings avatar Mar 31 '25 08:03 ilovedumplings

Image KjINCiQzDQpnZXQNCiQ0NA0KMjgyNThiYmEtMDNhYS00NjI5LTkwZDAtMTNiOGFlY2EzZWZiQVBJS0VZLTENCg==

ilovedumplings avatar Mar 31 '25 08:03 ilovedumplings

@johnlanni @rinfx 这个问题能协助看看嘛?

ilovedumplings avatar Mar 31 '25 09:03 ilovedumplings

Image

我复现不了你说的情况

rinfx avatar Apr 01 '25 11:04 rinfx

我推了个 higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-quota:1.0.1 的镜像，你改成这个试试

rinfx avatar Apr 01 '25 11:04 rinfx

我推了个 higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/ai-quota:1.0.1 的镜像，你改成这个试试

好我试试.1.0.1没问题 是不是latest镜像不更新了?我可能是一直用的latest镜像导致的

ilovedumplings avatar Apr 01 '25 12:04 ilovedumplings

@rinfx 你重新推一下 1.0.0 吧。我明天把 1.0.0 和 latest 同步一下。

CH3CHO avatar Apr 01 '25 14:04 CH3CHO

@rinfx 已经发布生产环境,然后发现了一个问题,就是刷新配额的时候超时..sha256开头为: c5665e5 查询是没有问题的

ilovedumplings avatar Apr 07 '25 04:04 ilovedumplings