Daft

Daft copied to clipboard

**Is your feature request related to a problem? Please describe.** I'd like to write directly to lance **Describe the solution you'd like** ```py df.write_lance('./lance_tbl') ``` **Describe alternatives you've considered** I...

universalmind303

Allow user to pass in Timestamp or Date Parsers

3

we should allow users to pass in Date / Timestamp format strings similar to how pyarrow takes in a list of timestamp parsers in the [convert options](https://arrow.apache.org/docs/python/generated/pyarrow.csv.ConvertOptions.html#pyarrow-csv-convertoptions).

samster25

**Describe the bug** **To Reproduce** using the same lineitem file found [here](https://github.com/Eventual-Inc/Daft/issues/2257#issuecomment-2118599555) ```sh aws s3 cp s3://daft-public-data/testing_data/bad-polars-lineitem.parquet ./lineitem.parquet --no-sign-request ``` ```py import daft (daft.read_parquet('.lineitem.parquet') .limit(100000) .write_parquet('lineitem.parquet')) ``` ``` DaftCoreException: DaftError::External...

universalmind303

parquet metadata only queries perform full scans

1

**Describe the bug** queries such as `read_parquet().count_rows()` should not do a full scan, and instead be should able to be fulfilled by the metadata only. The need for the full...

universalmind303

make commands fail on python 3.12

1

**Describe the bug** unable to run `make .venv` on python 3.12 **To Reproduce** ```sh > ❯ python3 --version Python 3.12.3 > make .venv python3 -m venv .venv .venv/bin/python -m pip...

universalmind303

[PyO3 version 0.21](https://github.com/PyO3/pyo3/releases/tag/v0.21.0) includes several things that we would find useful - the new Bound API that improves performance over GIL refs - support for datetime conversions with the abi3...

kevinzwang

[FEAT] add `list.value_counts()`

2

still a few things to do particular with types being null or not... but at least should not crash now TODO - [ ] add desc - [ ] make...

andrewgazelka

[CHORE] use cargo-machete in pre-commit hook

4

questions: - can we install it somehow? how about a pinned version? - which order should it be in? - am I doing this right? I am not familiar with...

andrewgazelka

[CHORE] add/fix many clippy lints

4

mostly auto-fix; a few manual fixes

andrewgazelka

# Map conversion fails for decimal types in test_all_types Iceberg table ## Description The `test_daft_iceberg_table_collect_correct` test is failing for the `test_all_types` table. The error occurs when trying to convert a...

andrewgazelka