Frederic Branczyk
Frederic Branczyk
I was pretty sparse on the initial implementation but the idea was that the rows in the results are the columns of the schema, so an additional column of this...
This would require two things: 1) Allow merging of schemas with complex types (with dynamic columns). 2) Extend conversion of complex (by complex I mean arbitrarily deeply nested struct and...
I wasn't aware of this paper, thanks for sharing @yeya24! I've only cross-read the paper, but from what I can tell what @albertlockett proposed so far is actually in line...
It's kind of a shame that `objstore` requires an `io.Reader` when all it does is copy that `io.Reader` to an `io.Writer`. Parquet files can be written in a single contiguous...
Ok, after looking ever so slightly into different providers, it seems that really only the GCS provider works the way I expected. I suspect the best way forward is to...
If we want to go the extra mile and avoid writing to disk at all cost we could propose an addition to the objstore API to get an `UploadWriter` which...
I think it's important that we keep the write path non-blocking/lock-free, so I wonder if we could find another way. Maybe we could just mark the granule to need compaction...
Potentially wild idea, what if we used a sync pool for reusing goroutines?
Never mind we would end up leaking goroutines.
Looks like we need to upgrade to go1.19, do you mind taking care of that @Mihai22125?