pkl icon indicating copy to clipboard operation
pkl copied to clipboard

File extension conflicts with Python's .pkl

Open bitnom opened this issue 1 year ago • 1 comments

I'm sure you're aware the file extension .pkl has long been used by Python. Think of all the Python scripts and tooling out there which look for .pkl files. One consequence off the top of my head is that many .gitignore boilerplates contain *.pkl. There must be countless projects already referencing .pkl files.

Maybe add a fourth character before this thing spreads any further. I'm super curious what the rationale was for greenlighting a conflicting file-extension.

bitnom avatar Feb 11 '24 16:02 bitnom

Per the Python Docs,

The pickle module is not secure. Only unpickle data you trust. ... Safer serialization formats such as json may be more appropriate if you are processing untrusted data. See Comparison with json

Thus, any programs searching the filesystem for *.pkl python files are responsible for maintaining that they only load .pkl files they trust, and not this format's config files.

In the other case that the program is not responsible for managing the directory, you'll also get thrown a _pickle.UnpicklingError: Memo value not found at [index] if attempting to unpickle a .pkl config file, which can be handled by the program.

Are you thinking of a project in particular for which this file extension conflict might cause an issue?

cellomath avatar Feb 11 '24 23:02 cellomath

As far as I'm aware, Python's pickle doesn't have an official file extension. Python's authors themselves seem to have moved on from .pkl, and now prefer .pickle: https://stackoverflow.com/questions/40433474/preferred-or-most-common-file-extension-for-a-python-pickle

For projects that look to use both Pkl and python pickle, maybe it's best to use .pickle as the filename for python pickle files.

bioball avatar Feb 14 '24 01:02 bioball

I don't think this is relevant. Pickled files have long been saved on developer preference; there's no particular reason that they have to use .pkl or .pickle.

chrisvander avatar Feb 14 '24 02:02 chrisvander

This should be closed as "not planned"

moritztim avatar Feb 14 '24 09:02 moritztim

Agree; closing this.

bioball avatar Feb 14 '24 15:02 bioball

I used pickle to save my model for a project. I stored the visualizer file also in pickle format. But when I am integrating it with Flask and using them to generate output. It is giving only 1 output. For the frontend I am using react. I have tried downloading model.pkl from colab but still getting wrong output. Can you please help me.

akshitsingh23 avatar Jul 07 '24 07:07 akshitsingh23

I used pickle to save my model for a project. I stored the visualizer file also in pickle format. But when I am integrating it with Flask and using them to generate output. It is giving only 1 output. For the frontend I am using react. I have tried downloading model.pkl from colab but still getting wrong output. Can you please help me.

@akshitsingh23 This is the comment section of an issue in a different project that uses the same file extension. This repository has nothing to do with python's pickle library. You could try directing your question at StackOverflow.

moritztim avatar Jul 08 '24 05:07 moritztim