openGemini icon indicating copy to clipboard operation
openGemini copied to clipboard
verified publisher icon openGemini

Continous query stops ts-sql when data retation is not created

Open Jafeyyu opened this issue 1 year ago • 0 comments

Describe the bug（Bug 描述）

创建连续查询去插入数据时，在指定写入数据的retention未创建的情况下，连续查询创建成功。当连续查询执行时会直接使ts-sql停止运行。具体情况是创建连续查询：

CREATE CONTINUOUS QUERY rpp_mean_data ON device BEGIN 
    SELECT max(floatValue) AS "floatValue" 
    INTO device.rpp_y1_data.jt2hl 
    FROM device.m1_data.jt2hl 
    WHERE "deviceName" =~ /列头/ 
    GROUP BY deviceName, deviceObjectName, deviceCode, deviceObjectCode, time(5m) TZ('Asia/Shanghai') 
END

但是存储策略rpp_y1_data并未创建。创建语句执行成功。当存储策略执行时，ts-sql停止运行，然后再次启动，存储策略执行时，ts-sql依旧会停止运行。

To Reproduce（Bug 复现步骤）

  1. 创建一个连续查询
CREATE CONTINUOUS QUERY rpp_mean_data ON device BEGIN 
    SELECT max(floatValue) AS "floatValue" 
    INTO device.rpp_y1_data.jt2hl 
    FROM device.m1_data.jt2hl 
    WHERE "deviceName" =~ /列头/ 
    GROUP BY deviceName, deviceObjectName, deviceCode, deviceObjectCode, time(5m) TZ('Asia/Shanghai') 
END

其中使用的存储策略rpp_y1_data未创建。

  1. 连续查询创建成功后，等待其执行

  2. ts-sql服务停止

查看sql_extra.log,内容如下：

panic: send on closed channel

goroutine 5490 [running]:
github.com/openGemini/openGemini/engine/executor.(*TargetTransform) .produce.func2()
    /home/runner/work/openGemini/openGemini/engine/executor/target transform.go:210 +0xab
github.com/openGemini/openGemini/lib/tracing(0xc001570f50?0xc001570f18?)
    /home/runner/work/openGemini/openGemini/lib/tracing/span.go: 183 +0x76*TargetTransform).produce(0xc001402c00, {0x196eee8, 0xc001297e50}, 0x0, 0xc001402c00?)
github.com/openGemini/openGemini/engine/executor.
    /home/runner/work/openGemini/openGemini/engine/executor/target transform,go:202 +0x18b
createdby qithub.com/openGemini/openGemini/engine/executor.(*TargetTransform).Work in goroutine 5405
    /home/runner/work/openGemini/openGemini/engine/executor/target transform.go: 172 +0x253

然后查看sql.log，会发现其中有error信息，此处只截取日志部分内容：

{"level":"info","time":"2024-05-13T17:53:32.07244136+08:00","msg":"Executing query","hostname":"10.8.152.197:8086","query":"SELECT max(floatValue) AS floatValue INTO device.rpp_y1_data.jt2hl FROM device.m1_data.jt2hl WHERE deviceName =~ /列头柜/ AND time >= '2024-05-13T09:45:00Z' AND time < '2024-05-13T09:50:00Z' GROUP BY deviceName, deviceObjectName, deviceCode, deviceObjectCode, time(5m) TZ('Asia/Shanghai')","location":"query/executor.go:375","repeated":3}
{"level":"info","time":"2024-05-13T17:53:33.906797041+08:00","msg":"10.8.152.197 - root [13/May/2024:17:53:33 +0800] \"POST /write?db=device&precision=n&consistency=all HTTP/1.1\" 204 0 \"-\" \"okhttp/4.12.0\" a946ffef-110e-11ef-8a4e-b414e683cf0c 373","hostname":"10.8.152.197:8086","location":"httpd/handler.go:2144","repeated":182}
{"level":"error","time":"2024-05-13T17:53:33.909233088+08:00","msg":"write point rows failed","hostname":"10.8.152.197:8086","db":"device","rp":"rpp_y1_data","error":"retention policy not found: rpp_y1_data","location":"coordinator/points_writer.go:240","repeated":1}
{"level":"info","time":"2024-05-13T17:54:48.81436811+08:00","caller":"syscontrol/syscontrol.go:153","msg":"DisableWrites","switch":false}
{"level":"info","time":"2024-05-13T17:54:48.814547425+08:00","caller":"syscontrol/syscontrol.go:158","msg":"DisableReads","switch":false}
{"level":"info","time":"2024-05-13T17:54:48.816901872+08:00","caller":"app/common.go:287","msg":"TSSQL starting","hostname":"10.8.152.197:8086","version":"v1.2.0","branch":"HEAD","commit":"eb398291ee88b7dde94f431a830e04544bd47e1f","buildTime":"2024-02-29T09:33:29Z"}
{"level":"info","time":"2024-05-13T17:54:48.816986628+08:00","caller":"app/common.go:292","msg":"Go runtime","hostname":"10.8.152.197:8086","version":"go1.21.7","maxprocs":52}
{"level":"info","time":"2024-05-13T17:54:50.480121987+08:00","caller":"syscontrol/syscontrol.go:153","msg":"DisableWrites","switch":false}
{"level":"info","time":"2024-05-13T17:54:50.480300764+08:00","caller":"syscontrol/syscontrol.go:158","msg":"DisableReads","switch":false}
{"level":"info","time":"2024-05-13T17:54:50.48103775+08:00","caller":"app/common.go:287","msg":"TSSQL starting","hostname":"10.8.152.197:8086","version":"v1.2.0","branch":"HEAD","commit":"eb398291ee88b7dde94f431a830e04544bd47e1f","buildTime":"2024-02-29T09:33:29Z"}
{"level":"info","time":"2024-05-13T17:54:50.481081775+08:00","caller":"app/common.go:292","msg":"Go runtime","hostname":"10.8.152.197:8086","version":"go1.21.7","maxprocs":52}
{"level":"info","time":"2024-05-13T17:54:50.822153697+08:00","caller":"httpd/service.go:108","msg":"Starting HTTP service","hostname":"10.8.152.197:8086","service":"httpd","authentication":false}
{"level":"info","time":"2024-05-13T17:54:50.822377902+08:00","caller":"httpd/service.go:100","msg":"Listening on HTTP","hostname":"10.8.152.197:8086","service":"httpd","addr":"10.8.152.197:8086","https":false}
{"level":"info","time":"2024-05-13T17:54:50.822677794+08:00","msg":"opened HTTP access log","hostname":"10.8.152.197:8086","path":"stderr","location":"httpd/handler.go:399","repeated":1}
{"level":"info","time":"2024-05-13T17:54:50.83328366+08:00","msg":"Starting continuousQuery enforcement service","hostname":"10.8.152.197:8086","check_interval":1,"location":"services/base.go:50","repeated":1}
{"level":"info","time":"2024-05-13T17:54:51.828769328+08:00","msg":"10.8.152.197 - root [13/May/2024:17:54:51 +0800] \"POST /write?db=device&precision=n&consistency=all HTTP/1.1\" 204 0 \"-\" \"okhttp/4.12.0\" d7b352d7-110e-11ef-8412-b414e683cf0c 40389","hostname":"10.8.152.197:8086","location":"httpd/handler.go:2144","repeated":106}
{"level":"info","time":"2024-05-13T17:54:51.829899246+08:00","msg":"continuous query lease changed","hostname":"10.8.152.197:8086","location":"continuousquery/service.go:199","repeated":1}
{"level":"info","time":"2024-05-13T17:54:51.830073141+08:00","msg":"current host has already registered in ts-meta","hostname":"10.8.152.197:8086","location":"metaclient/meta_client.go:3393","repeated":1}
{"level":"info","time":"2024-05-13T17:54:51.837108413+08:00","msg":"Executing query","hostname":"10.8.152.197:8086","query":"SELECT mean(floatValue) AS floatValue INTO device.m3_data.jt2hl FROM device.m1_data.jt2hl WHERE time >= '2024-05-13T09:45:00Z' AND time < '2024-05-13T09:50:00Z' GROUP BY deviceName, deviceObjectName, deviceCode, deviceObjectCode, time(5m) TZ('Asia/Shanghai')","location":"query/executor.go:375","repeated":3}
{"level":"info","time":"2024-05-13T17:54:52.486364009+08:00","caller":"httpd/service.go:108","msg":"Starting HTTP service","hostname":"10.8.152.197:8086","service":"httpd","authentication":false}
{"level":"error","time":"2024-05-13T17:54:52.486533984+08:00","msg":"failed to start http server","hostname":"10.8.152.197:8086","addr":"10.8.152.197:6061","location":"sql/server.go:264","repeated":1}
{"level":"info","time":"2024-05-13T17:54:53.334010316+08:00","msg":"10.8.152.194 - root [13/May/2024:17:54:53 +0800] \"POST /write?db=device&precision=n&consistency=all HTTP/1.1\" 204 0 \"-\" \"okhttp/4.12.0\" d89f1ba8-110e-11ef-8989-b414e683cf0c 357","hostname":"10.8.152.197:8086","location":"httpd/handler.go:2144","repeated":144}
  1. 启动ts-sql，在连续查询执行之前，删除此连续查询后，ts-sql恢复正常

Expected behavior（期望结果）

创建连续查询，如果使用的存储策略未创建或者语句有问题，只是在日志中进行错误日志的输出，不影响ts-sql或者其他服务的正常运行

Screenshots（屏幕截图）

微信图片_20240513184237 微信图片_20240513184328

微信截图_20240513184259

Logs（完整的错误日志）

[sql.error.log](https://github.com/openGemini/openGemini/files/15293234/sql.error.log) [sql.log](https://github.com/openGemini/openGemini/files/15293235/sql.log)

Additional context（其他的一些补充内容）

No response

Jafeyyu avatar May 13 '24 10:05 Jafeyyu