ekuiper icon indicating copy to clipboard operation
ekuiper copied to clipboard

memory source transform stream make problem

Open rthink-hyf opened this issue 3 years ago • 2 comments

Environment: ekuiper version 1.4.1

What happened and what you expected to happen:

External Function rule (select getData(chanId, fileName) as res from dsp_dataReq) save result to memory source interface result format:[{"res":{"data":[{"name":"x","resolution":0.1}]}}] stream format(from memory): res struct(data array(struct(name string, resolution float)) the error is shown below

中文描述一下就是: 我把定义了一个外部函数的调用规则,然后把调用结果放到内存源中。接口的返回格式如下:[{"res":{"data":[{"name":"x","resolution":0.1}]}}] 然后再创建一个stream去订阅该内存源,stream的格式如下:res struct(data array(struct(name string, resolution float)) 结果就报错了,报一个interface{}类型的转换错误: image

Another question about memory source The bigint type of stream cannot be converted to the int32 or int64 type in proto, but float can. This feels unreasonable stream的bigint类型,与proto中的int32或int64类型无法相互转换,但float是可以的。这个感觉不合理

rthink-hyf avatar Mar 03 '22 06:03 rthink-hyf

@rthink-hyf If you have fixed this problem, we welcome a PR 🥇

ngjaying avatar Mar 04 '22 08:03 ngjaying

Do not have time in this release, so defer

ngjaying avatar Apr 01 '22 03:04 ngjaying

POST http://{{host}}/streams

{"sql":"create stream demo_stream (res struct(data array(struct(name string, resolution float))),) 
WITH ( datasource = \"demo\", format = \"json\")"}

###

GET http://{{host}}/streams/mem_stream

###

POST http://{{host}}/rules

{
  "id": "rule1",
  "sql": "SELECT * FROM demo_stream",
  "actions": [{
    "log": {
    },
    "memory": {
      "topic": "memory/demo"
    }
  }]
}

###

GET http://{{host}}/rules/rule1

###

POST http://{{host}}/streams

{"sql":"create stream mem_stream (res struct(data array(struct(name string, resolution float))),)
WITH ( datasource = \"memory/demo\", format = \"json\", type = \"memory\", STRICT_VALIDATION = \"true\")"}

###

Created test streams and rules, but cannot reproduce the same interface panic error.

2023-06-21 10:53:03 time="2023-06-21 02:53:03" level=info msg="sink result for rule rule1: [{\"res\":{\"data\":[{\"name\":\"x\",\"resolution\":0.1}]}}]" file="sink/log_sink.go:32" rule=rule1
2023-06-21 10:54:15 time="2023-06-21 02:54:15" level=info msg="sink result for rule rule1: [{\"res\":{\"data\":[{\"name\":\"x\",\"resolution\":0.1}]}}]" file="sink/log_sink.go:32" rule=rule1
2023-06-21 10:54:16 time="2023-06-21 02:54:16" level=info msg="sink result for rule rule1: [{\"res\":{\"data\":[{\"name\":\"x\",\"resolution\":0.1}]}}]" file="sink/log_sink.go:32" rule=rule1

Conversion seems to be successful.

xjasonlyu avatar Jun 21 '23 03:06 xjasonlyu

It seems like fixed along the version updates, so close. Reopen it if you still find problems.

ngjaying avatar Jun 21 '23 09:06 ngjaying