logstash-output-clickhouse

logstash-output-clickhouse copied to clipboard

Reference: https://www.altinity.com/blog/2017/12/18/logstash-with-clickhouse

Hello Andrey

I am trying to make the table name dynamic by passing a value which is combination of JSON fields from data being ingested. For instance:

clickhouse {
                        http_hosts => ["http://172.17.0.5:8123"]
                        user => "default"
                        password => "password"
                        # table => "w46a1d81.srstream" 
                        table => "%{stream}-%{profile_id}"
                        request_tolerance => 1
                        flush_size => 1000
                        pool_max => 1000
                        mutations => {
                                "stream" => "stream"
                                "profile_id" => "profile_id"
                                "user_id" => "user_id"
                               ....
                        }
                }

This set-up when executed throws error. Ingestion URLs are constructed like:

http://172.17.0.5:8123/?query=INSERT%20INTO%20%{stream}-%{profile_id}%20FORMAT%20JSONEachRow

Any ideas how this can be solved? Does this feature exists or we need to build it?