python-logging-loki
python-logging-loki copied to clipboard
Improve performance with batching
Currently it only uploads one log line with each HTTP request, which does not work well for high velocity logs.
The queued handler could potentially read all the available lines in the queue and upload everything outstanding at once as opposed to popping an item one at a time, and uploading it
This would be nice feature, otherwise I'm little bit afraid to use it in production on high-load web servers. Did you come to any solution @theblazehen ?
@tomashornak Afraid not, decided to just log to journald with https://pypi.org/project/systemd-logging/ and use promtail from there, but haven't tested it at scale yet
@tomashornak Afraid not, decided to just log to journald with https://pypi.org/project/systemd-logging/ and use promtail from there, but haven't tested it at scale yet
How are you propagating unique_labels, tags etc in your solution?