PSeitz

> There's no way of determining that one? I thought length of the block is stored somewhere in the meta as well. Yes, each block stores it's length. The terminator...

You could write multiple frames instead of a frame with multiple blocks. Then load the complete frames and discard the last corrupted one.

We have a limit currently after which don't merge segments. This limit does not take into account multi values in columns, which could exceed the value space `u32`. As a...

I think yes, it seems toasted. How many multi values per doc on average does your column have?

So there are no arrays in your data? In that case, the issue is probably still similar, but not sure what exactly causes it.

The `LogMergePolicy` only checks if a single segment does not exceed 10 Mio docs, but not that a group does not exceed the global threshold of 1 billion. ```rust let...

I couldn't reproduce with current nightly (2024-12-17) or stable (1.83.0 (90b35a623 2024-11-26)) on macos

Can confirm, this fixes the issue

> I've managed to do this for my personal site with `{{ load_data(path="path/to/svg.svg") | safe }}`. > > You can check out the source code here: [timharek/timharek.no:/layouts/templates/base/header.html](https://github.com/timharek/timharek.no/blob/main/templates/base/header.html#L3) Is there a...

```json { "query": "*", "max_hits": 0, "aggs": { "comment_ranking_avg": { "terms": { "field": "comment_ranking", "size": 20, "order": { "avg_field": "desc", "_key": "desc" } }, "aggs": { "avg_field": { "avg": {...