polars icon indicating copy to clipboard operation
polars copied to clipboard

Can't read ndjson file with null object

Open pebble2050 opened this issue 2 years ago • 1 comments

Polars version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of Polars.

Issue description

Can't read ndjson file with null object. Simplar to #6075 But need another solve approach, Since json file may have too many fileds, passing type as paramter may be not accepable.

Reproducible example

import polars as pl
from io import BytesIO

nj = "{\"a\":1,\"b\":{}}\n{\"a\":2,\"b\":{}}\n"

buf = BytesIO(bytes(nj,'ascii'))

df = pl.read_ndjson(buf)
print(df)

Expected behavior

read normally

Installed versions

---Version info---
Polars: 0.15.14
Index type: UInt32
Platform: Windows-10-10.0.19045-SP0
Python: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)]
---Optional dependencies---
pyarrow: 10.0.1
pandas: 1.5.2
numpy: 1.22.3
fsspec: 2022.11.0
connectorx: <not installed>
xlsx2csv: <not installed>
matplotlib: 3.5.2

pebble2050 avatar Jan 10 '23 07:01 pebble2050

@universalmind303 could you take a look at this one?

ritchie46 avatar Jan 20 '23 13:01 ritchie46