PrometheusAlert icon indicating copy to clipboard operation
PrometheusAlert copied to clipboard

告警组不生效,ddurl1,ddurl2 告警推送报错

Open dellnoantechnp opened this issue 1 year ago • 1 comments

当前版本:镜像tag:master 最新版

关键配置:

open-dingding=1
ddurl1=https://oapi.dingtalk.com/robot/send?access_token=c6f01194cxxxxx&secret=SECxxxxx094
ddurl2=https://oapi.dingtalk.com/robot/send?access_token=abf0df1b9xxxxx&secret=SECxxxxxxxdab
open-dingding-secret=1
open-alertgroup=1

[ag-demo]
wxurl=wxurl1,wxurl2
ddurl=ddurl1,ddurl1,
fsurl=fsurl1
email=email1,
phone=phone1,phone2
groupid=groupid1

[ag-alert]
ddurl=ddurl1,ddurl2
# include "alertgroup.conf"

alertmanager receiver 配置:

receivers: 
  - name: my-prometheusAlert
    webhookConfigs:
      - url: "http://my-prometheusalert.prometheus.svc:8080/prometheusalert?type=dd&tpl=prometheus-dd&alertgroup=alert&split=false"       <--- 使用 alert 组的方式进行通知
        sendResolved: true
        httpConfig:
          followRedirects: true

如上配置过后,在 PrometheusAlert 日志中报错:

2024/06/27 15:14:06.264 [D] [prometheusalert.go:409]  SplitString alertname%3D%22PrometheusRuleFailures%22%2Ccontainer%3D%22prometheus%22%2Cendpoint%3D%22http-web%22%2Cinstance%3D%22172.25.71.25
3:9090%22%2Cjob%3D%22kube-prometheus-stack-prometheus%22%2Cnamespace%3D%22prometheus%22%2Cpod%3D%22prometheus-kube-prometheus-stack-prometheus-0%22%2Cprometheus%3D%22prometheus/kube-prometheus-s
tack-prometheus%22%2Cservice%3D%22kube-prometheus-stack-prometheus%22%2Cseverity%3D%22critical%22%2C
2024/06/27 15:14:06.264 [I] [dingding.go:42]  [dingdingSign] 配置文件已开启钉钉加签,钉钉机器人地址解析加签参数 secret 为空,将使用不加签的地址!
2024/06/27 15:14:06.264 [I] [prometheusalert.go:456]  [1719501246263154212] [dingding] {"msgtype":"markdown","markdown":{"title":"PrometheusAlert","text":"\r\n\r\n---\r\n\r\n##  \u003cfont color
=\"#FF0000\"\u003e**Prometheus 告警通知**\u003c/font\u003e\r\n* 🟡 【告警名称】PrometheusRuleFailures\r\n*  .............
2024/06/27 15:14:06.264 [E] [prometheusalert.go:456]  [1719501246263154212] [dingding] Post "": unsupported protocol scheme ""   
2024/06/27 15:14:06.264 [C] [panic.go:884]  the request url is  /prometheusalert                                                                                                                  
2024/06/27 15:14:06.264 [C] [panic.go:884]  Handler crashed with error runtime error: invalid memory address or nil pointer dereference                                                           
2024/06/27 15:14:06.264 [C] [panic.go:884]  /usr/local/go/src/runtime/panic.go:884                                                                                                                
2024/06/27 15:14:06.264 [C] [panic.go:884]  /usr/local/go/src/runtime/panic.go:260                                                                                                                
2024/06/27 15:14:06.265 [C] [panic.go:884]  /usr/local/go/src/runtime/signal_unix.go:841                                                                                                          
2024/06/27 15:14:06.265 [C] [panic.go:884]  /go/src/github.com/feiyu563/PrometheusAlert/controllers/dingding.go:95                                                                                
2024/06/27 15:14:06.265 [C] [panic.go:884]  /go/src/github.com/feiyu563/PrometheusAlert/controllers/prometheusalert.go:456                                                                        
2024/06/27 15:14:06.265 [C] [panic.go:884]  /go/src/github.com/feiyu563/PrometheusAlert/controllers/prometheusalert.go:210                                                                        
2024/06/27 15:14:06.265 [C] [panic.go:884]  /usr/local/go/src/reflect/value.go:586                                                                                                                
2024/06/27 15:14:06.265 [C] [panic.go:884]  /usr/local/go/src/reflect/value.go:370                                                                                                                
2024/06/27 15:14:06.265 [C] [panic.go:884]  /go/src/github.com/feiyu563/PrometheusAlert/vendor/github.com/astaxie/beego/router.go:897                                                             
2024/06/27 15:14:06.265 [C] [panic.go:884]  /usr/local/go/src/net/http/server.go:2936                                                                                                             
2024/06/27 15:14:06.265 [C] [panic.go:884]  /usr/local/go/src/net/http/server.go:1995                                                                                                             
2024/06/27 15:14:06.265 [C] [panic.go:884]  /usr/local/go/src/runtime/asm_amd64.s:1598                                                                                                            
2024/06/27 15:14:06.265 [server.go:3215]  [HTTP] http: superfluous response.WriteHeader call from github.com/astaxie/beego/context.(*Response).WriteHeader (context.go:230)

ddurl 处尝试过将 &secret 替换为 %26secret 字符,一样的报错。


另外当把配置 ddurl1 改为 ddurl 时,保留 ddurl2 的通知地址配置。 配置变更为如下格式:

ddurl=https://oapi.dingtalk.com/robot/send?access_token=c6f01194cxxxxx&secret=SECxxxxx094
ddurl2=https://oapi.dingtalk.com/robot/send?access_token=abf0df1b9xxxxx&secret=SECxxxxxxxdab
open-dingding-secret=1
open-alertgroup=1

[ag-alert]
ddurl=ddurl,ddurl2

我便只会在 ddurl 对应的通知群里收到对应的告警消息,日志无异常信息

请问这部分是否有做适配? 还是只在不加签的状况内测试通过? 又或者这部分逻辑与 webhook 上所配置的 url 包含 alertgroup=alert 逻辑有冲突?

dellnoantechnp avatar Jun 27 '24 15:06 dellnoantechnp

@Zhang21 大佬,有碰到过这个问题吗?

dellnoantechnp avatar Jun 27 '24 15:06 dellnoantechnp

告警组适配钉钉加签名,我测试过的。

alertgroup=alert,你的配置里没有叫 alert 这个组,我看你的配置的告警组名称是 ag-alert 。

Zhang21 avatar Jul 01 '24 01:07 Zhang21

你配置写的不对,ddurl 只是 webhook 地址的缩写,并不是变量名称。

[告警组名称]
ddurl=https://地址1不加签,https://地址2&签名

Zhang21 avatar Jul 01 '24 01:07 Zhang21