How to restrain memory use of insert into queries

[ChenZhaobin]

Everytime when I add an insert into query ,the memory will be incremented by 0.08GB，even if there is no data coming throught the stream, I need 20000 insert into queries in my case ,so do you guys have better idea? please help me

[agavra]

Hey @ChenZhaobin can you clarify what you mean by "20000 insert into queries"? Are you trying to running 20k concurrent INSERT INTO ... statements, are there 20k events going through a single INSERT INTO or are you inserting 20K events via INSERT INTO ... VALUES statements?

It would be good to get a sense of your use case so we can help you dig into this.

[ChenZhaobin]

Hey @agavra , running 20k concurrent INSERT INTO ... statements ,this is what I mean. I followed this post: https://www.confluent.io/blog/bounding-ksqldb-memory-usage/ , but it still is not possible to create 20k concurrent INSERT INTO ... statements . In my case ,I need to create different streams (or create multiple insert into statements into a same stream ) based on different duration. such ascreate stream ....within Duration, and Duration is a variable, if we set Duration 20k times ,then we will need 20k concurrent INSERT INTO ...` statements

[nigulable]

Hi @ChenZhaobin did you figure out your problem? I dont have anywhere near 20k, but unsure how to limit memory - that link does not seem to be fixing our problem either