polars icon indicating copy to clipboard operation
polars copied to clipboard

write_parquet and sink_parquet delete existing file if locked

Open Lerofni opened this issue 1 month ago • 2 comments

Checks

  • [x] I have checked that this issue has not already been reported.
  • [x] I have confirmed this bug exists on the latest version of Polars.

Reproducible example


import polars as pl
from filelock import FileLock
import time



data = {
    "col1": [1,2,3],
    "col2": [4,5,6],
}
test = pl.DataFrame(data)
test.write_parquet('test.parquet')
time.sleep(2)
test = test.select(pl.col("col1"))
lock = FileLock('test.parquet')
with lock:
    test.write_parquet('test.parquet')

Log output


Issue description

Due to the Lock Polars returns a ComputeError and deletes the file, even if it was completely valid.

Expected behavior

The file should not be deleted if polars cannot touch it

Installed versions

--------Version info---------
Polars:              1.35.2
Index type:          UInt32
Platform:            Windows-10-10.0.22631-SP0
Python:              3.11.9 (tags/v3.11.9:de54cf5, Apr  2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)]
Runtime:             rt32
----Optional dependencies----
Azure CLI            <not installed>
'az' is not recognized as an internal or external command,
operable program or batch file.
adbc_driver_manager  <not installed>
altair               <not installed>
azure.identity       <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            0.14.0
fsspec               <not installed>
gevent               <not installed>
google.auth          2.40.3
great_tables         <not installed>
matplotlib           <not installed>
numpy                2.2.6
openpyxl             3.1.5
pandas               2.2.3
polars_cloud         <not installed>
pyarrow              21.0.0
pydantic             <not installed>
pyiceberg            <not installed>
sqlalchemy           <not installed>
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           <not installed>

Lerofni avatar Nov 19 '25 14:11 Lerofni

I can't reproduce on MacOS. You didn't follow the issue template and showed us the output of pl.show_versions().

orlp avatar Nov 20 '25 15:11 orlp

I have added the output of pl.show_versions()

Lerofni avatar Dec 03 '25 09:12 Lerofni