polars icon indicating copy to clipboard operation
polars copied to clipboard

`.write_json()` with a struct column causes a segfault

Open cmdlineluser opened this issue 2 years ago • 2 comments

Polars version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of Polars.

Issue description

.write_json() on a dataframe containing a struct column is causing python to segfault.

Setting json_lines=True or using .write_ndjson() work as expected.

Reproducible example

import polars as pl

pl.DataFrame({"a": [{"b": "c"}]}).write_json()

# Segmentation fault: 11

Expected behavior

Not to segfault.

Installed versions

---Version info---
Polars: 0.15.15
Index type: UInt32
Python: 3.10.9 (main, Jan 14 2023, 16:56:26)
---Optional dependencies---
pyarrow: 9.0.0
pandas: 1.5.3
numpy: 1.23.5
fsspec: 2022.8.2
connectorx: <not installed>
xlsx2csv: 0.8
deltalake: <not installed>
matplotlib: 3.5.1

cmdlineluser avatar Jan 20 '23 22:01 cmdlineluser

Valgrind says it is a stackoverflow, so I assume we recurse somewhere.

ritchie46 avatar Jan 21 '23 14:01 ritchie46

It looks like the issue is with serde_json::to_writer()

https://github.com/pola-rs/polars/blob/master/py-polars/src/dataframe.rs#L395 seems to be codepath taken.

cmdlineluser avatar Jan 21 '23 17:01 cmdlineluser