prometheus-webhook-dingtalk icon indicating copy to clipboard operation
prometheus-webhook-dingtalk copied to clipboard

Published 20 hours ago • verified publisher icon timonwong

unexpected status code 404

Open kerven88 opened this issue 4 years ago • 13 comments

alertmanager配置了dingtalk地址后，日志看到请求404，请问这个是什么原因？

image

kerven88 avatar Jun 09 '20 11:06 kerven88

在配置文件中这样配置 prometheus-webhook-dingtalk.yml

targets:
  webhook:
    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxxxxxxxxxxxx
    secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
template: dingtalk-message.tmpl

启动prometheus-webhook-dingtalk加上参数

/usr/local/bin/prometheus-webhook-dingtalk --config.file=prometheus-webhook-dingtalk/prometheus-webhook-dingtalk.yml --web.enable-lifecycle

之后你应该可以在日志中看到

Jun 10 09:47:11 192-168-1-xxx prometheus-webhook-dingtalk[20176]: level=info ts=2020-06-10T01:47:11.122Z caller=main.go:117 component=configuration msg="Loading templates" templates=/etc/prometheus-webhook-dingtal component=configuration msg="Loading templates" templates=templates/dingtalk-message.tmpl

Jun 10 09:47:11 192-168-1-xxx prometheus-webhook-dingtalk[20176]: ts=2020-06-10T01:47:11.124Z caller=main.go:133 component=configuration msg="Webhook urls for prometheus alertmanager" urls=http://localhost:8060/dingtalk/webhook/send

is-a-gamer avatar Jun 10 '20 01:06 is-a-gamer

Customizable templates path

templates:

  • contrib/templates/legacy/template.tmpl

我这样配置模板，reload时候提示格式问题。 是要按照你上面方式配置吗？ template: dingtalk-message.tmpl

kerven88 avatar Jun 10 '20 02:06 kerven88

另外，在prometheus-webhook-dingtalk里面配置了tmpl模板之后，是会覆盖掉alertmanager配置的模板吗？

kerven88 avatar Jun 10 '20 02:06 kerven88

例

template: /etc/prometheus/templates/dingtalk-message.tmpl

template: templates/dingtalk-message.tmpl

如果提示模板文件格式错误,先检查模板文件 可以使用default.tmpl尝试去启动

在prometheus-webhook-dingtalk里面配置了tmpl模板之后,通过prometheus-webhook-dingtalk发送到钉钉信息,模板采用的应该是prometheus-webhook-dingtalk的

is-a-gamer avatar Jun 10 '20 02:06 is-a-gamer

模板dingtalk.tmpl [root@prometheus prometheus-webhook-dingtalk]# cat dingtalk.tmpl {{ define "__subject" }}[{{ .Annotations.summary }}]{{ end }} {{ define "__text_list" }}{{ range . }} {{ range .Alerts }}

实例: {{ .Labels.instance }}
信息: {{ .Annotations.summary }}
详情: {{ .Annotations.description }}
时间: {{ .StartsAt.Format "2006-01-02 15:04:05" }}

{{ end }} {{ end }}

但是发送钉钉后的信息没有按这个模板来发。 image

服务日志是正常的： image 帮忙看看哪里有问题？

kerven88 avatar Jun 10 '20 02:06 kerven88

如果仍然是默认的信息格式的话 请重新拉取代码并重新构建 约40分钟前有一个"When specifying a template, the default template is no longer used"的合并 修复了不能使用自定义模板的BUG

is-a-gamer avatar Jun 10 '20 03:06 is-a-gamer

重新拉取代码编译后，启动有报错： image Jun 10 11:27:33 prometheus prometheus-webhook-dingtalk[467]: level=error ts=2020-06-10T03:27:33.160Z caller=dingtalk.go:90 component=web target=webhook msg="Failed to build notification" err="template: :1:12: executing "" at <{{template "ding.link.title" .}}>: template "ding.link.title" not defined"

配置还是沿用之前版本的配置，之前版本启动正常。

kerven88 avatar Jun 10 '20 03:06 kerven88

之前版本启动正常是因为实际上加载的是"default.tmpl",而现在代码已经修正,会加载你指定的模板文件,所以实际上你的配置文件还是不正确的 你可以试着在模板中加上这一行 (在上面看到了你已经定义了"__subject")

{{ define "ding.link.title" }}{{ template "__subject" . }}{{ end }}

is-a-gamer avatar Jun 10 '20 03:06 is-a-gamer

加了这行配置，还是报错： Jun 10 11:45:33 prometheus prometheus-webhook-dingtalk[1736]: level=error ts=2020-06-10T03:45:33.154Z caller=dingtalk.go:90 component=web target=webhook msg="Failed to build notification" err="template: :1:12: executing "" at <{{template "ding.link.title" .}}>: template "ding.link.title" not defined" Jun 10 11:45:33 prometheus prometheus-webhook-dingtalk[1736]: level=info ts=2020-06-10T03:45:33.154Z caller=entry.go:22 component=web http_scheme=http http_proto=HTTP/1.1 http_method=POST remote_addr=127.0.0.1:54592 user_agent=Alertmanager/0.20.0 uri=http://localhost:8060/dingtalk/webhook/send resp_status=400 resp_bytes_length=12 resp_elapsed_ms=0.769532 msg="request complete"

dingtalk.tmpl配置如下： image

kerven88 avatar Jun 10 '20 03:06 kerven88

那我们来试试最简单的

{{ define "ding.link.title" }}测试标题{{ end }}
{{ define "ding.link.content" }}测试内容{{ end }}

这个时候你的钉钉消息应该只会显示"测试内容"

is-a-gamer avatar Jun 10 '20 04:06 is-a-gamer

用了你这个tmpl，还是一样的报错： Jun 10 12:28:03 prometheus prometheus-webhook-dingtalk[5906]: level=error ts=2020-06-10T04:28:03.240Z caller=dingtalk.go:90 component=web target=webhook msg="Failed to build notification" err="template: :1:12: executing "" at <{{template "ding.link.title" .}}>: template "ding.link.title" not defined" Jun 10 12:28:03 prometheus prometheus-webhook-dingtalk[5906]: level=info ts=2020-06-10T04:28:03.240Z caller=entry.go:22 component=web http_scheme=http http_proto=HTTP/1.1 http_method=POST remote_addr=127.0.0.1:56664 user_agent=Alertmanager/0.20.0 uri=http://localhost:8060/dingtalk/webhook/send resp_status=400 resp_bytes_length=12 resp_elapsed_ms=1.440891 msg="request complete"

image

kerven88 avatar Jun 10 '20 04:06 kerven88

@kerven88 建议你先使用 v1.4.0

#105 有一个 breaking change，就是要求 template 必须定义完成所有的内容，但是最初这个设计是分层的（也就是一层一层的覆盖），类似 prometheus 本身的 template。所以当你加载 legacy/template.tmpl 的时候，会发现缺乏一些定义模板

第二个就是，模板其实也不用非得是文件，可以直接写到 yaml 里面的，比如:

#.... 省略
targets:
  webhook_legacy:
    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxx
    # Customize template content
    # 这里利用了 yaml multiline string
    message:
      title: |-
        {{ template "subject" }}
      text: |-
        测试内容

建议开个 UI 测试: https://github.com/timonwong/prometheus-webhook-dingtalk/blob/master/docs/FAQ_zh.md#%E5%A6%82%E4%BD%95%E7%BC%96%E5%86%99%E6%A8%A1%E6%9D%BF

timonwong avatar Jun 10 '20 14:06 timonwong

在配置文件中这样配置 prometheus-webhook-dingtalk.yml

targets:
  webhook:
    url: https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxxxxxxxxxxxx
    secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
template: dingtalk-message.tmpl

启动prometheus-webhook-dingtalk加上参数

/usr/local/bin/prometheus-webhook-dingtalk --config.file=prometheus-webhook-dingtalk/prometheus-webhook-dingtalk.yml --web.enable-lifecycle

之后你应该可以在日志中看到

Jun 10 09:47:11 192-168-1-xxx promethe么us-webhook-dingtalk[20176]: level=info ts=2020-06-10T01:47:11.122Z caller=main.go:117 component=configuration msg="Loading templates" templates=/etc/prometheus-webhook-dingtal component=configuration msg="Loading templates" templates=templates/dingtalk-message.tmpl

Jun 10 09:47:11 192-168-1-xxx prometheus-webhook-dingtalk[20176]: ts=2020-06-10T01:47:11.124Z caller=main.go:133 component=configuration msg="Webhook urls for prometheus alertmanager" urls=http://localhost:8060/dingtalk/webhook/send

我怎么看不到这个

level=info ts=2020-07-03T11:33:20.053Z caller=main.go:62 msg="Starting prometheus-webhook-dingtalk" version="(version=1.4.0, branch=HEAD, revision=02fe8265a98ab4caaa78ebbed209d3f06b87b4a6)" level=info ts=2020-07-03T11:33:20.053Z caller=main.go:63 msg="Build context" (gogo1.13.5,userroot@eb9f8d8f0437,date20191211-03:00:38)=(MISSING) level=info ts=2020-07-03T11:33:20.053Z caller=coordinator.go:89 component=configuration file=/prometheus-webhook-dingtalk/prometheus-webhook-dingtalk.yml msg="Loading configuration file" level=info ts=2020-07-03T11:33:20.054Z caller=coordinator.go:97 component=configuration file=/prometheus-webhook-dingtalk/prometheus-webhook-dingtalk.yml msg="Completed loading of configuration file" level=info ts=2020-07-03T11:33:20.054Z caller=main.go:117 component=configuration msg="Loading templates" templates=dingtalk-message.tmpl ts=2020-07-03T11:33:20.054Z caller=main.go:133 component=configuration msg="Webhook urls for prometheus alertmanager" urls=http://localhost:8060/dingtalk/webhook/send level=info ts=2020-07-03T11:33:20.054Z caller=web.go:210 component=web msg="Start listening for connections" address=:8060 level=warn ts=2020-07-03T11:33:28.625Z caller=dingtalk.go:75 component=web target=webhook1 msg="target not found" level=info ts=2020-07-03T11:33:28.625Z caller=entry.go:22 component=web http_scheme=http http_proto=HTTP/1.1 http_method=POST remote_addr=172.16.0.17:40148 user_agent=Alertmanager/0.15.2 uri=http://10.247.175.67:8060/dingtalk/webhook1/send resp_status=404 resp_bytes_length=19 resp_elapsed_ms=0.054328 msg="request complete" level=warn ts=2020-07-03T11:33:38.621Z caller=dingtalk.go:75 component=web target=webhook1 msg="target not found" level=info ts=2020-07-03T11:33:38.621Z caller=entry.go:22 component=web http_scheme=http http_proto=HTTP/1.1 http_method=POST remote_addr=172.16.0.17:40204 user_agent=Alertmanager/0.15.2 uri=http://10.247.175.67:8060/dingtalk/webhook1/send resp_status=404 resp_bytes_length=19 resp_elapsed_ms=0.072233 msg="request complete" level=warn ts=2020-07-03T11:33:48.621Z caller=dingtalk.go:75 component=web target=webhook1 msg="target not found" level=info ts=2020-07-03T11:33:48.621Z caller=entry.go:22 component=web http_scheme=http http_proto=HTTP/1.1 http_method=POST remote_addr=172.16.0.17:40254 user_agent=Alertmanager/0.15.2 uri=http://10.247.175.67:8060/dingtalk/webhook1/send resp_status=404 resp_bytes_length=19 resp_elapsed_ms=0.062649 msg="request complete" level=warn ts=2020-07-03T11:33:58.621Z caller=dingtalk.go:75 component=web target=webhook1 msg="target not found" level=info ts=2020-07-03T11:33:58.621Z caller=entry.go:22 component=web http_scheme=http http_proto=HTTP/1.1 http_method=POST remote_addr=172.16.0.17:40306 user_agent=Alertmanager/0.15.2 uri=http://10.247.175.67:8060/dingtalk/webhook1/send resp_status=404 resp_bytes_length=19 resp_elapsed_ms=0.063835 msg="request complete"

模版是要单独定义的么？

reaperhero avatar Jul 03 '20 11:07 reaperhero