Andrew Duffy

Results 35 comments of Andrew Duffy

Just chiming in, I'm also hitting this issue in the context of trying to store the tonic client inside of my own struct. ``` pub struct SchemaClient { inner: SchemaServiceClient...

We also bumped into this, it appears to be fixed since the merge in upstream Spark by using `ArrayList` instead of `Array` https://github.com/apache/spark/commit/96030876383822645a5b35698ee407a8d4eb76af#diff-01fea32e6ec6bcf6f34d06282e08705aR103

Also seeing this, here is an example failed GH Action run in case helpful https://github.com/IntrinsicLabsAI/intrinsic-model-server/actions/runs/6868629622/job/18679613918

Bumping my Python patch release from 3.11.4 -> 3.11.5 fixed the issue for me. It looks like Poetry may have fixed this via a (as yet unreleased) backport: https://github.com/python-poetry/poetry/pull/8649/files So,...

Hey! Just curious, are there any plans to add the filters to the C API? I have a file format (implemented in Rust) that I'd like to write an extension...

![image](https://github.com/user-attachments/assets/f7f909e8-529d-4e79-a2d5-c24085f51093) Ran the compress_taxi benchmark, got ~80% slower. I am a bit surprised that the biggest culprit seems to be creating new counters in the FSST training loop. That doesn't...

Alright, using the change in https://github.com/spiraldb/fsst/pull/21 helped a lot. New benchmark result: ``` end to end - taxi/compress time: [100.73 ms 101.72 ms 102.96 ms] change: [-45.073% -43.470% -42.057%] (p...

And I think we can go even lower, ideally we'd just use the trained compressor over the samples to compress the full array

Ok I've done a few things today 1. Introduced a way to reuse compressors in our samplingcompressor code 2. Keep tweaking some things on the FSST side, including matching how...

Ok I added a new benchmark now which just compresses the comments column in-memory via Vortex, and i'm seeing it take ~500ms, which is roughly 2-3x longer than just doing...