fastparquet
fastparquet copied to clipboard
dask
python implementation of the parquet columnar file format.
**What happened**: Error when working with a pandas.DataFrame that has been loaded from a partitioned parquet file. **What you expected to happen**: Non corrupted DataFrame. **Minimal Complete Verifiable Example**: ```python...
**What happened**: Checking a test case, I see that if some partition values are `None`, corresponding row group is not created. See test `test_groups_roundtrip` in `test_output.py`. ```python def test_groups_roundtrip(tempdir, scheme):...
Until 0.7.0 pandas `datetime` column by default converted into int64, TIMESTAMP_MICROS parquet type. It seems like [this statement](https://github.com/dask/fastparquet/blob/0.7.0/fastparquet/writer.py#L156) (`if "ns" in dtype.str`) changes that default behaviour. It is also not...
**Environment**: - Python version: 3.7.10 - Pip version: 19.3.1 - Operating System: Using Google Colab - Install method (conda, pip, source): pip **Description**: Unable to import fastparquet library in a...
In our application we mainly append new data to a parquet dataset, say when a user uploads content to our application. Sometimes however a user wants to close his account...
When doing a row-wise filtering (not row-group-wise filtering) with filters '>', '>=', '=', '
Future Plans
A list of nice-to-haves that could be implemented in fastparquet. The order is arbitrary. I am writing this as a placeholder, and for people to express interest in the features...
Hi, Do you think we could record in the metadata the size of the row groups when recording them? This data can be obtained when dataframe is split in row_group...
**What happened**: Reading a parquet file to a pandas dataframe, fastparquet crashes due to an error in unpack_byte_array. I cannot share the parquet file due to PII. It has been...
Encountered error "no module named 'fastparquet.cencoding' " after using fastparquet-0.7.1-cp39-cp39-manylinux2014_aarch64.whl file in aws glue
