Ziheng Wang

quantized models

1

Does this support quantized models by any chance?

ARROW-17313: [C++][Python] Add fragment.slice() for CSV file format

5

This is going to be a draft PR based on discussions in https://issues.apache.org/jira/browse/ARROW-17313. This PR addresses the CSV format. The Parquet format I believe is covered by already implemented API...

ARROW-17299: [C++][Python] Expose the Scanner kDefaultBatchReadahead and kDefaultFragmentReadahead parameters

6

This exposes the Fragment Readahead and Batch Readahead flags in the C++ Scanner to the user in Python. This can be used to finetune RAM usage and IO utilization during...

ARROW-14635: [Python][C++] implement fadvise

4

After: https://github.com/apache/arrow/pull/13640, seems like O_DIRECT is not a good idea, so let's use posix_fadvise to control the page cache to address the issue mentioned in: https://issues.apache.org/jira/browse/ARROW-14635. To test it, use...

Remove Redis as a dependency

1

The current data plane uses Ray for same machine comms and Redis for inter-machine comms. Both should be replaced by a custom Arrow Flight server, with two benefits: - No...

How do we track progress and get estimated job completion time with Quokka? In Quokka we have input actors whose progress can easily be tracked (we know the upper limit...

Any plans?

ARROW-17481: [C++][Python] Major performance improvements to CSV reading from S3

4

I messed up the Git rebase again: https://github.com/apache/arrow/pull/13931 So let's now look at this PR instead.

This needs to happen at some point. Issues are static linking apache arrow. ideally we just copy a binary. I guess we could also copy a binary and the arrow...