Switch field in the influxdb3_wal crate to use ColumnId instead of a Arc<str>
This is a follow up to #25388. We want to use ColumnIds instead of Arc
https://github.com/influxdata/influxdb/blob/219fc168ead075cebe64b7559345e497340d0b12/influxdb3_wal/src/lib.rs#L574
@pauldix this is turning out to not be a simple change and has far reaching implications. For instance:
- The BufferIndex depends on the name of the column, especially when working with Exprs which are from Datafusion
- A lot of the times we need names, it's because of Datafusion
- LastCache expects names for lookups, especially when interfacing with, you guessed it, the Datafusion Schema type
I think we'd spend more time doing a conversion lookup then just having an Arc
We could still try to make it work, but this is not a single day task like I had originally hoped for and I'll likely need to pick this up after vacation or we reconsider the value of ColumnId at all. Even the wrapper TableSchema is proving hard to pass around, when so much of the code really only cares about a schema::Schema type. I'm worried we're adding a lot of overhead for very little gain here.
I think it's worth doing mainly because that goes into the wal constantly, many more times than you'd do the lookup for queries. Ideally the last cache and queryable buffer would keep the data indexed by ID too. Just doing conversion on the read side when pulling data out.
Definitely not urgent so it can wait a while.
Okay I'll pick this up when I get back from vacation then!