higress icon indicating copy to clipboard operation
higress copied to clipboard

proxy wasm redis client 应该在请求 redis 有问题时打印错误日志

Open Colstuwjx opened this issue 10 months ago • 3 comments

Why do you need it?

在 macOS M1 本地通过 kind 拉起了一套 higress 集群,redis 资源拉起来后发现连不上,见相关 issue ,那么假定 redis 服务有问题的话,应该打印出连接失败的日志,并且在请求 redis 失败时应该报错。前者是可以看到日志的:

2025-03-04T13:12:16.834973Z	critical	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1404	wasm log higress-system.ai-token-ratelimit: Error occured while calling redis, it seems cannot connect to the redis cluster. request-id: 7753f680-eeda-4849-9edf-a40b6347f10e	thread=53
2025-03-04T13:12:16.835056Z	error	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1401	wasm log higress-system.ai-token-ratelimit: [ai-token-ratelimit] [nil] [048282ff-a612-4b97-9903-d7f2487c7ff4] redis response parse error, response: cannot connect to redis cluster	thread=53

但是通过在 ai-token-ratelimit plugin 里打印日志,发现 eval 请求(具体代码位置)还会实际去执行,而且只会打印一条 start 的日志没有后续执行出错或执行结束的日志:

2025-03-04T13:12:18.518203Z	info	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1395	wasm log higress-system.ai-token-ratelimit: [ai-token-ratelimit] [nil] [048282ff-a612-4b97-9903-d7f2487c7ff4] onHttpStreamingBody eval redis keys [higress-token-ratelimit:default_limit_by_header_beartoken:limit_by_per_header:Authorization:Bearer XXX], args [1 60 18]	thread=53
2025-03-04T13:12:18.518505Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1392	wasm log higress-system.ai-token-ratelimit: redis call start, request-id: 471ebb4f-33e5-4a03-9e23-c7d40c84ffe1, respQuery: XXX	thread=53

How could it be?

应当有一条 redis call failed 之类的错误日志,有注意到实际调用 redis 时是通过设置一个 callback 方法来执行后续的处理。

但是看了下 redis client 具体的实现部分,是否是这里有问题?如果回复状态不是 internal.StatusOK 而是连都没连上的话应该就无法执行 callback 方法了?还请帮忙确认下。

Other related information

macOS M1, Kind cluster.

Colstuwjx avatar Mar 04 '25 13:03 Colstuwjx

cc @rinfx 把 http 调用相关的日志优先级也一并调整一下吧

johnlanni avatar Mar 08 '25 07:03 johnlanni

这个是因为目前在处理streamingbody时无法通过返回action.pause等待redis调用完成,所以没有添加回调函数

rinfx avatar Mar 17 '25 09:03 rinfx

@johnlanni @rinfx 看看我这里做的实现可不可以。可以的话,我就提个 PR 上来。

https://github.com/alibaba/higress/issues/1810#issuecomment-2709248860

CH3CHO avatar Mar 17 '25 11:03 CH3CHO