Paul Dix comments

Results 77 comments of


                                            Paul Dix

Create an Official Golang client for InfluxDB3 Core/Enterprise

Pulling in @garylfowler as he'll be the most likely one to know our timeline for this

E2E integration tests are flaky

I think I like the log option

Have snapshots continuously retry in the background

@praveen-influx is this still relevant after all the snapshot work you recently did?

Have snapshots continuously retry in the background

We'll close this as won't fix as it won't be relevant when the new storage engine arrives.

Avoid OOM when running snapshots at higher throughput

For the part where it breaks down the data and produces smaller Parquet files. When doing this within a table chunk, it should make sure that the resulting Parquet files...

Avoid OOM when running snapshots at higher throughput

You can break an individual chunk down into multiple parts by looping through the time array and figuring out where you want to do the split. All the other columns...

Avoid OOM when running snapshots at higher throughput

The problem with changing the chunk time in the queryable buffer and then not changing it in the WAL is that the snapshot logic could be wrong. I'd have to...

Avoid OOM when running snapshots at higher throughput

I'm not sure it's worth the extra effort to optimize that at this moment. We should come back to it, but I think this can probably be closed.

metrics: Processing Engine

We don't expect that many triggers to be defined. What does it look like if they have 1k triggers (an exceptionally high number)?

Switch field in the influxdb3_wal crate to use ColumnId instead of a Arc<str>

I think it's worth doing mainly because that goes into the wal constantly, many more times than you'd do the lookup for queries. Ideally the last cache and queryable buffer...