Keep translator comments next to the translation function call

[akx]

(even if the text is further away)

Fixes #1195.

[akx]

@stitch-martha Would you be able to test the code from this PR to see if it fixes #1195 for you?

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.72%. Comparing base (d7a7589) to head (0a73e4a). Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1196   +/-   ##
=======================================
  Coverage   91.71%   91.72%           
=======================================
  Files          27       27           
  Lines        4685     4687    +2     
=======================================
+ Hits         4297     4299    +2     
  Misses        388      388           

Flag	Coverage Δ
macos-14-3.10	90.76% <100.00%> (+<0.01%)	:arrow_up:
macos-14-3.11	90.69% <100.00%> (+<0.01%)	:arrow_up:
macos-14-3.12	90.91% <100.00%> (+<0.01%)	:arrow_up:
macos-14-3.13	90.91% <100.00%> (+<0.01%)	:arrow_up:
macos-14-3.8	90.62% <100.00%> (+<0.01%)	:arrow_up:
macos-14-3.9	90.69% <100.00%> (+<0.01%)	:arrow_up:
macos-14-pypy3.10	90.76% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-3.10	90.78% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-3.11	90.71% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-3.12	90.93% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-3.13	90.93% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-3.8	90.64% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-3.9	90.71% <100.00%> (+<0.01%)	:arrow_up:
ubuntu-24.04-pypy3.10	90.56% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-3.10	90.77% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-3.11	90.70% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-3.12	90.92% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-3.13	90.92% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-3.8	90.74% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-3.9	90.70% <100.00%> (+<0.01%)	:arrow_up:
windows-2022-pypy3.10	90.77% <100.00%> (+<0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[stitch-martha]

@akx I have installed babel from this git branch and ran the command and the comments are still getting deleted unfortunately. For reference I am running the command with pybabel extract <path> --output-file=<file> --copyright-holder <holder> --project <project-name> --add-comment='NOTE' --no-location --no-wrap

[akx]

@stitch-martha Hmm...

Given a 1195.py like

x = {
        'test_string': StringWithMeta(
        # NOTE: Text describing a test string
        string=_(
            'Text string that is on a new line'
        ),
    ),
}

, running pybabel extract --output-file=1195.po --copyright-holder holder --project project --add-comment='NOTE' --no-location --no-wrap 1195.py from this code yields

[...]
"Generated-By: Babel 2.17.0\n"

#. NOTE: Text describing a test string
msgid "Text string that is on a new line"
msgstr ""

[stitch-martha]

Actually the short text works for me too, but only because Black formats it onto one line...

Text string that is on a new line

@akx What if you change to this 'Text string that is on a new line and is very very long so that black does not format it'?

[akx]

Text string that is on a new line and is very very long so that black does not format it

x = {
        'test_string': StringWithMeta(
        # NOTE: Text describing a test string
        string=_(
            'Text string that is on a new line and is very very long so that black does not format it'
        ),
    ),
}

👇

#. NOTE: Text describing a test string
msgid "Text string that is on a new line and is very very long so that black does not format it"
msgstr ""

Works for me, even in a fresh Docker container on this branch 🤔

$ docker run -it -v (pwd):/app ghcr.io/astral-sh/uv:python3.13-bookworm-slim bash
# cd /app
# uv venv /tmp/venv
Using CPython 3.13.2 interpreter at: /usr/local/bin/python
Creating virtual environment at: /tmp/venv
# source /tmp/venv/bin/activate
(venv) # uv pip install -e .
Using Python 3.13.2 environment at: /tmp/venv
Resolved 1 package in 1.02s
      Built babel @ file:///app
Prepared 1 package in 429ms
Installed 1 package in 1ms
 + babel==2.17.0 (from file:///app)
(venv) # pybabel extract --output-file=1195.po --copyright-holder holder --project project --add-comment='NOTE' --no-location --no-wrap 1195.py && cat 1195.po
[...]
"Generated-By: Babel 2.17.0\n"

#. NOTE: Text describing a test string
msgid "Text string that is on a new line and is very very long so that black does not format it"
msgstr ""

---

EDIT:

        # NOTE: Text describing a test string
        string=_(
            'Text string that is on a new line and is very very '
            'long and now split onto many lines '
            'in a poetic manner'
        ),

also does the right thing.

[stitch-martha]

Thanks for checking

I've realised that in my test I wasn't correctly using this brach's code, I was still using 2.17.0, which explains why I was still hitting the bug.

I have now cloned the repo so that I could use this version directly into my project but I am struggling to build it, when I run make import-cldr I am getting an SSL error:

running import_cldr
Local copy '/Users/mbowler/Desktop/Developer/babel/cldr/cldr-common-46.0.zip' not found
Downloading 'cldr-common-46.0.zip' from https://unicode.org/Public/cldr/46/cldr-common-46.0.zip
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1298, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1344, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1293, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1052, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 990, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/http/client.py", line 1470, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1382, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mbowler/Desktop/Developer/babel/scripts/download_import_cldr.py", line 88, in <module>
    main()
  File "/Users/mbowler/Desktop/Developer/babel/scripts/download_import_cldr.py", line 64, in main
    urlretrieve(URL, tmp_path, (reporthook if show_progress else None))
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 241, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
                            ^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)>
Traceback (most recent call last):
  File "/Users/mbowler/Desktop/Developer/babel/setup.py", line 27, in <module>
    setup(
  File "/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/lib/python3.11/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/Users/mbowler/Desktop/Developer/babel/setup.py", line 24, in run
    subprocess.check_call([sys.executable, 'scripts/download_import_cldr.py'])
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/mbowler/Desktop/Developer/trialmap/trialmap-messenger/venv/bin/python', 'scripts/download_import_cldr.py']' returned non-zero exit status 1.

[akx]

That's probably your (corporate?) environment's proxy interfering with an SSL certificate we don't know how to trust.

As a workaround, you can just copy the locale-data directory from your installed babel (or from a 2.17.0 release wheel's contents – they're just zips) to your working directory.

[stitch-martha]

@akx Thank you for the suggestion - I have managed to do that and the fix is working!