cpython icon indicating copy to clipboard operation
cpython copied to clipboard

gh-125756: Document `Pickler.clear_memo`

Open tomasr8 opened this issue 1 year ago • 3 comments

Documents the public clear_memo which is currently undocumented. I think it makes to have it in the docs given that it is need when reusing Pickler objects.

The documentation is a copy of the docstring, but perhaps we could make it even clearer by replacing This method is useful when re-using picklers. with something like When reusing picklers, this method should be called after every call to dump?

  • Issue: gh-125756

📚 Documentation preview 📚: https://cpython-previews--125762.org.readthedocs.build/

tomasr8 avatar Oct 20 '24 22:10 tomasr8

I hope you fix the problems with clear_memo before making this doc public Did you read my comment about it on the original isue?

erik895 avatar Oct 23 '24 10:10 erik895

I hope you fix the problems with clear_memo before making this doc public Did you read my comment about it on the original isue?

The function is public already unfortunately, the documentation is not the source of truth, the code is.

FFY00 avatar Oct 23 '24 15:10 FFY00

The documentation is a copy of the docstring, but perhaps we could make it even clearer by replacing This method is useful when re-using picklers. with something like When reusing picklers, this method should be called after every call to dump?

That sounds good.

We should also warn the users about potential misuse, as pointed out by @erik895

FFY00 avatar Oct 23 '24 15:10 FFY00

I hope you fix the problems with clear_memo before making this doc public Did you read my comment about it on the original isue?

Sorry for the late reply, had a busy week :/ As Filipe pointed out, the function is public so there is not much we can do besides documenting its gotchas.

I saw your example, though I think it's a pretty rare case to pickle multiple objects separately rather than just pickling one structure (list, dictionary, what have you..) and of course if you use clear_memo you can break things but there's no way to prevent someone from doing that if they really want to..

How about we add this note to the docs?

This method is useful when re-using picklers, however it should be avoided when calling dump() repeatedly with the same file as it can cause unpickle() to load incorrect objects.

tomasr8 avatar Oct 25 '24 19:10 tomasr8

Thanks @tomasr8 for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12. 🐍🍒⛏🤖

miss-islington-app[bot] avatar Feb 17 '25 15:02 miss-islington-app[bot]

Thanks @tomasr8 for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. 🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington-app[bot] avatar Feb 17 '25 15:02 miss-islington-app[bot]

GH-130231 is a backport of this pull request to the 3.12 branch.

bedevere-app[bot] avatar Feb 17 '25 15:02 bedevere-app[bot]

GH-130232 is a backport of this pull request to the 3.13 branch.

bedevere-app[bot] avatar Feb 17 '25 15:02 bedevere-app[bot]

Well, I am not sure about backporting the documentation. Usually this is not an issue, but one of options is to deprecate this function -- in that case it is not to document it in older versions.

serhiy-storchaka avatar Feb 17 '25 15:02 serhiy-storchaka