Akumuli icon indicating copy to clipboard operation
Akumuli copied to clipboard

Support for data ingestion over HTTP(S) with optional gzip compression.

Open dmitrios opened this issue 6 years ago • 1 comments

While TCP and UDP are supported, consider adding HTTP transport for receiving data. With Gzip compression using InfluxDb average packet size is reduced by 50% for data with repeatable groups. In addition, some frameworks do not expose performance efficient TCP socket pool directly, but do support it well using HTTP client APIs. Using HTTPS would guarantee privacy of data sent outside DMZ to time series database.

dmitrios avatar Mar 28 '18 17:03 dmitrios

Hi, You may be interested in this - https://github.com/akumuli/Akumuli/pull/262 This PR added the ability to send a dictionary with series names and user provided ids. This ids can be used instead of series names later on (as long as TCP connection lives). It makes the transferred data smaller (reduced by 50% in my experiments, from 5088344490 bytes to just 2405943390). This may already help to reduce the transfer size. I'm looking forward to adding HTTP ingestion in the future though. But it will take some time.

Lazin avatar Mar 29 '18 08:03 Lazin