Özgür Akkurt

@ritchie46 currently arrow binary datatype isn't supported. not implemented here: https://github.com/pola-rs/polars/blob/4beb08eba86bfcf8924d1d484fccb78b94c0e48e/polars/polars-core/src/datatypes/field.rs#L113 also here: https://github.com/pola-rs/polars/blob/4beb08eba86bfcf8924d1d484fccb78b94c0e48e/polars/polars-core/src/datatypes/dtype.rs#L6

Hey, I'm working on this

maybe can use [async_std](https://docs.rs/async-std/latest/async_std/index.html)::[task](https://docs.rs/async-std/latest/async_std/task/index.html)::[spawn_blocking](https://docs.rs/async-std/latest/async_std/task/fn.spawn_blocking.html#) or convert io code to use `async`?

Hey @sehz! Tracking how async-std handles thread count limits: - https://github.com/async-rs/async-std/blob/bf316b095c176c8738c6401cc62d0bc389c88961/src/rt/mod.rs#L12 - https://github.com/async-rs/async-global-executor/blob/0abe723db4ad440f5cebbd06f95b6234a8116398/src/config.rs#L62 - https://doc.rust-lang.org/stable/std/thread/fn.available_parallelism.html So it seems to use [this](https://doc.rust-lang.org/stable/std/thread/fn.available_parallelism.html) by default. And it seems to be controllable by...

Yeah seems like it. I'll try to do this

> > so we must decode to a pointer of u8s first and then trasmute > > I'm worried about alignment issues in such an approach. > > > What...

thinking more about this, can have a third function `get_fingerprint_indices(&self, key: &[u8]) -> (u64, u64, u64, u64)` that only returns where the fingerprints should be located so the downstream application...

> @ozgrakkurt did you want to take a stab at putting up a pr implementing it? Thinking more about it, I'll just use blocked bloom filter since I can essentially...

@vlovich can use something like this (I didn't test it but it should work fine): ```rust use std::collections::BTreeMap; use futures::StreamExt; use glommio::io::{ImmutableFile, MergedBufferLimit, ReadAmplificationLimit}; use anyhow::{Context, Result, anyhow}; #[inline(always)] pub...

@vlovich I just got this into the code and it makes a huge difference. Thanks for great idea! When scanning all indices in db, it made it about 5x as...