mxlxm

Why don't we just remove the manually partition assignment? Set group.id with the same ID across multiple indexer, thus consumergroup bases partition assignment can do the same things for us

> The first one is that we want to handle more sources than just kafka, and some of these source do not offer the concept of consumer group. Kinesis for...

https://fridge.ubuntu.com/2022/06/01/ubuntu-21-10-impish-indri-reaches-end-of-life-on-july-14-2022/

I did some work with VM-1.64.1-cluster, fixed some bugs: - memory leak in vmselect - broken labels api - corrupted logs within vmstorage - wrong direction in vmselect - dead...

Loki's great for lightweight environment, simple and more efficient than es. But yes, it's unusable with heavy load(peek at 60w logs/s). The pipeline design sucks with huge data. So, I'm...

I've tried Loki-distributed(with memcache enabled) with the same data, basically unable to query for entry rate large than 1w/s, even within 15 minutes. And querier would consume endless memory without...

I'll send the PR later. By the way, I think we shouldn't change `value` type in vmstorage, how about extend vmstorage with a new field `data` to store log data,...

@faceair https://github.com/faceair/VictoriaLogs/pull/7 PR is ready for review.

https://github.com/faceair/VictoriaLogs/pull/7/commits/c287aab9681a7ad6800842510439d96beb6b4076 can revert with next vm release, since upstream has fixed, see https://github.com/VictoriaMetrics/VictoriaMetrics/pull/1597

> I think we shouldn't change value type in vmstorage, how about extend vmstorage with a new field data to store log data, each log's value defaults to 1, so...