接入 TDEngine 后的 grafana 表达式

[xxj123go]

prometheus 写入 TDEngine 后，原有几百个grafana 表达式都需要改成 类sql 形式，这个有工具可以处理吗，数量太多了

[sangshuduo]

很好的建议，我们评估一下

[zitsen]

@xxj123go 参考Bailongma的README，Prometheus接入TDengine需要同时接入remote read/write ，Grafana不需要直接接入TDengine，按照原有的方式和表达式接Prometheus就可以，Bailongma会处理promql到TDengine sql的转换。

[xxj123go]

@zitsen 按照你说的试了试，可以，但如果采集点很多，blm_prometheus 会不会成为瓶颈。期间还遇到 blm_prometheus 异常， 2021/09/09 15:55:16 http: panic serving 10.204.11.124:52150: interface conversion: interface {} is time.Time, not string goroutine 326 [running]: net/http.(*conn).serve.func1(0xc00019e1e0) /usr/local/go/src/net/http/server.go:1824 +0x153 panic(0xa821a0, 0xc00086a180) /usr/local/go/src/runtime/panic.go:971 +0x499 bailongma/v2/blm_prometheus/pkg/tdengine/read.(*ReaderProcessor).Process(0xc000161980, 0xc00000e0a8, 0x0, 0x0, 0x0) /usr/local/go/src/bailongma/v2/blm_prometheus/pkg/tdengine/read/reader.go:86 +0x15c7 bailongma/v2/blm_prometheus/pkg/tdengine.(*Client).Read(0x11bce38, 0xc00000e0a8, 0xe, 0xc000055a98, 0x1) /usr/local/go/src/bailongma/v2/blm_prometheus/pkg/tdengine/client.go:72 +0x35 main.readHandle.func1(0xbff158, 0xc0002801c0, 0xc000464d00) /usr/local/go/src/bailongma/v2/blm_prometheus/server.go:241 +0x4bf net/http.HandlerFunc.ServeHTTP(0xc0000b07c8, 0xbff158, 0xc0002801c0, 0xc000464d00) /usr/local/go/src/net/http/server.go:2069 +0x44 net/http.(*ServeMux).ServeHTTP(0x118e140, 0xbff158, 0xc0002801c0, 0xc000464d00) /usr/local/go/src/net/http/server.go:2448 +0x1ad net/http.serverHandler.ServeHTTP(0xc000280000, 0xbff158, 0xc0002801c0, 0xc000464d00) /usr/local/go/src/net/http/server.go:2887 +0xa3 net/http.(*conn).serve(0xc00019e1e0, 0xc002c0, 0xc0006af6c0) /usr/local/go/src/net/http/server.go:1952 +0x8cd created by net/http.(*Server).Serve /usr/local/go/src/net/http/server.go:3013 +0x39b

简单定位发现这个异常和 读操作有关，总是伴随着如下 log 一块产生： cat /var/log/taos/blm_prometheus.log
[INFO] 2021/09/09 15:54:56 server.go:239: req info== {[start_timestamp_ms:1631173794494 end_timestamp_ms:1631174094494 matchers:<name:"__name__" value:"up" > hints:<start_ms:1631173794494 end_ms:1631174094494 > ]} [INFO] 2021/09/09 15:54:56 reader.go:204: startTime：1631173794494 ,endTime:1631174094494 [INFO] 2021/09/09 15:54:56 reader.go:50: Executed query：SELECT * FROM up WHERE ts >= 1631173794494 AND ts <= 1631174094494 ORDER BY ts 同样，在prometheus web端查询也可以触发blm_prometheus进程抛上述异常

进一步关联到 /bailongma/v2/blm_prometheus/pkg/tdengine/read.go ，麻烦帮忙进一步确认。