Python 3.13.0b3 REPL empties ~/.python_history

[hroncok]

Bug report

Bug description:

As discussed in https://discuss.python.org/t/python-3-13-0-beta-3-now-available/56847/7 and https://github.com/python/cpython/issues/120766#issuecomment-2200251720

When I updated from Python 3.13.0b2 to b3 and started the REPL for the first time, pressing arrow up showed the history.

Upon exiting and starting again, there is no more history when I press arrow up. I noticed it is also missing from my other Python REPLs. ~/.python_history is empty.

To reproduce, I did:

1. run python3.13
2. press arrow up, hit enter
3. Ctrl+D
4. run python3.13 again -- no history

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-121255
• gh-121259
• gh-121261
• gh-121322

[hroncok]

This is always reproducible:

$ python3.12
Python 3.12.3 (main, Apr 17 2024, 00:00:00) [GCC 13.2.1 20240316 (Red Hat 13.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 'Save this'
'Save this'
>>> ^D

$ cat ~/.python_history 
'Save this'

$ python3.13
Python 3.13.0b3 (main, Jun 28 2024, 00:00:00) [GCC 13.3.1 20240522 (Red Hat 13.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> [arrow up]'Save this'
'Save this'
>>> ^D

$ cat ~/.python_history 
(empty)

$ python3.12
Python 3.12.3 (main, Apr 17 2024, 00:00:00) [GCC 13.2.1 20240316 (Red Hat 13.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> [arrow up]
(empty)

[hroncok]

@domdfcoding wrote:

... The first time Python 3.13 runs the history is there, but after exiting and running again the history has gone. Running an older Python populates the history with new entries but they're lost the next time Python 3.13 exits.

Git bisect tracked the issue to 38cfa92. The actual issue is in site.py, where it tries to import CAN_USE_PYREPL from __main__, which it's no longer located in as of that commit.

https://github.com/python/cpython/blob/c7991cc28788bbb086fd85d8fc55e20742f0de88/Lib/site.py#L527-L532

Changing this to from _pyrepl.main import CAN_USE_PYREPL or adding from .main import CAN_USE_PYREPL to the top of _pyrepl.__main__.py seems to fix it. (I couldn't make it work reliably at first but I think that was me forgetting that site.py is frozen)

[hroncok]

Moreover, I believe that even if the code in site.py "cannot use pyrepl", it should never ever write empty history.

[skirpichev]

PR is ready: https://github.com/python/cpython/pull/121255

[hroncok]

Suggestion: Let's merge https://github.com/python/cpython/pull/121255 as is but do not close this issue yet. I think there are multiple improvements possible:

• [ ] add a regression test
• [ ] prevent empty history write
• [ ] make write_history() more symmetrical wrt reading the history file (see https://github.com/python/cpython/pull/121255#issuecomment-2202359457)

[skirpichev]

Yep, I'll work on this in second pr.

[skirpichev]

Second PR is ready for review: https://github.com/python/cpython/pull/121259

[skirpichev]

See #121351. @hroncok, I think that means that simple patch from the first PR doesn't fix problem on all systems.

Probably we should use my patch 8195067 from https://github.com/python/cpython/pull/121259. To make it work we can set CAN_USE_REPL before loading the site module, see https://github.com/python/cpython/pull/121259#issuecomment-2205060209

[skirpichev]

PR is ready for review: https://github.com/python/cpython/pull/121659

[skirpichev]

@ambv, probably this also could be closed.

[picnixz]

As this is not directly related to make test in particular, I'll actually keep https://github.com/python/cpython/issues/139352 instead. Sorry for the noise.