Andrew K

Results 6 comments of Andrew K

This is the error I am seeing for this issue ```python Broken DAG: [/dags/airflow-db-cleanup.py] Traceback (most recent call last): File "xxx/lib/python3.9/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "xxx/lib/python3.9/json/encoder.py",...

I was able to get a small file written without error but I still encounter the error with a large dataset. I re-ran on a different system with pandas 2.2.1...

If there is only one Dataframe partition saving works fine - if there is > 1 partition, this error is returned.

Thanks for that idea @Hoxbro. I downgraded dask to 2020 but it returns the same error. So far in looking into the issue I found that any call to `df.geometry.total_bounds`...

I did try fastparquet (same error). However, I don't think it's related to that or to saving directly. Something happens with pack_partitions that causes and future calls to the geometry.total_bounds...

I found a trigger condition for the error - it occurs when one or more longitudes are negative. I attached a simple notebook that reproduces the error. If you change...