Matt Corley

Results 38 comments of Matt Corley

Many, many +1s for the 2 big ideas here (dynamically generating specs for a kernel + ability to run lsp per kernel). It's really hard to adopt this extension usefully...

@jkleinkauff Without profiling it's hard to say for sure, but I can make a bet based on my experience with object storage clients/fsspec implementations in particular: - you're benchmarking the...

@kevinjqliu I did find a pure-python approach that is faster (~2.4x on my machine) than `pyarrow.array(range(...))`: ```python import pyarrow as pa import ctypes def create_arrow_range(start: int, end: int) -> pa.Array:...

Just wanted to echo that this would be incredibly useful for scenarios I face in my current work as well (mainly in helping onboard analysts to ibis, while still allowing...

@gforsyth I am so excited for this work! Thanks for the progress on it.

I'd add one thing, possibly out of scope for this RFC: it would be really nice to be able to define a "max execution concurrency" on the backend, either propeller-wide...

> execution namespace quota is meant to help address quality of service and fairness in a multitenant system but it would be cool to flesh out other mechanisms for managing...

@charliermarsh I know it's painful/stupid/messy, but the regex assignment is still really useful for handling transitive dependencies in corporate environments. I don't think there are easy alternatives given the default...

I am really curious about how `Load View` should work, given that currently only SQL [representations ](https://iceberg.apache.org/view-spec/#summary)of views are supported and I don't think we have an in-process SQL engine...

I agree with @sungwy that the primary goal of pyiceberg should be to make it possible for query engines to interface with Iceberg tables and views. Nonetheless, it would be...