httprb
Support HTTP headers streaming
Clickhouse able to send query progress in header during query execution. Documentation Example (no anchor, just find send_progress_in_http_headers
When we're making a request, we're getting all progress headers in the end of query (when query finished and we got the response) How can we read stream of headers during execution?
We can't provide a real example, but that's how you can reproduce it with curl if you have some clickhouse node to access:
echo "SOME LONG QUERY WITH PROGRESS" | curl -vvv -XPOST 'clickhouse_node:8123/?user=USER&send_progress_in_http_headers=1&query=' --data-binary @-
> POST /?user=USER&send_progress_in_http_headers=1&query= HTTP/1.1
> Host: clickhouse_node:8123
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Length: 425
> Content-Type: application/x-www-form-urlencoded
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 16 Nov 2023 14:10:50 GMT
< Connection: Keep-Alive
< Content-Type: text/tab-separated-values; charset=UTF-8
< Transfer-Encoding: chunked
< X-ClickHouse-Query-Id: QUERY_ID
< X-ClickHouse-Format: TabSeparated
< X-ClickHouse-Timezone: Zulu
< Keep-Alive: timeout=3
< X-ClickHouse-Progress: {"read_rows":"65661","read_bytes":"7794203","written_rows":"0","written_bytes":"0","total_rows_to_read":"47703090","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"2983003","read_bytes":"355204021","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"6136877","read_bytes":"729970296","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"9207044","read_bytes":"1094719900","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"12303523","read_bytes":"1462010911","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
< X-ClickHouse-Progress: {"read_rows":"15088486","read_bytes":"1794558825","written_rows":"0","written_bytes":"0","total_rows_to_read":"5267326958","result_rows":"0","result_bytes":"0"}
As you can see above, curl allow to stream headers in realtime.
If it really needed, we can setup some dockerized example.
We don't currently support this, only streaming bodies.
It might be possible to add but would require careful API considerations.
I think this is something we want though, in order to control response consumption.
