Niels Bantilan

Results 459 comments of Niels Bantilan
trafficstars

Hi @pthalasta looking at the [avro schema docs](https://avro.apache.org/docs/1.11.1/getting-started-python/) it looks like we'll need to write a translation layer between `avro -> pandera`, similar to the frictionless integration: https://pandera.readthedocs.io/en/stable/frictionless.html?highlight=frictionless#frictionless-data-schema Feel free...

@sam-goodwin feel free to make a PR to add the generic types [here](https://github.com/unionai-oss/pandera/blob/main/pandera/typing/common.py#L55) and [here](https://github.com/unionai-oss/pandera/blob/main/pandera/typing/common.py#L97). Also, FYI you can use the type without `pdt.Series` ```python int32: pdt.list[pdt.Int32] = pa.Field() ```

@rorymcstay thanks! need to run pre-commit to fix linter errors: https://pandera.readthedocs.io/en/stable/CONTRIBUTING.html#set-up-pre-commit

Thanks @sam-goodwin, see https://pandera.readthedocs.io/en/latest/CONTRIBUTING.html#set-up-pre-commit for steps to make sure linters and unit tests are passing. You'll also need to sign your commits: https://pandera.readthedocs.io/en/latest/CONTRIBUTING.html#dco-signing-commits

Mypy errors: ``` tests/core/test_typing.py:498: error: "list" is not subscriptable, use "typing.List" instead [misc] tests/core/test_typing.py:499: error: "dict" is not subscriptable, use "typing.Dict" instead [misc] tests/core/test_typing.py:500: error: "tuple" is not subscriptable, use...

> Do you mean we can't specify ints with specific precision in a List or Dict in pandera? This just follows the way pandas deals with data. Columns containing `list`...

@sam-goodwin friendly ping: one of the unit tests is still failing: https://github.com/unionai-oss/pandera/actions/runs/8861081819/job/24332580434?pr=1556

@mattharrison I think this would be a feature request: the current scope of pandera is that it doesn't yet support pyarrow datatypes/backend. Gonna close https://github.com/unionai-oss/pandera/issues/1162 and merge that with this...

Anyone who wants to create PR for this has my blessing! A good place to start would be: - Dtype docs: https://pandera.readthedocs.io/en/stable/dtypes.html - Pandas engine implementation for datatypes: https://github.com/unionai-oss/pandera/blob/main/pandera/engines/pandas_engine.py