Duplicate lines in the wheel RECORD file when a native extension is built

[SpecLad]

• Poetry version: 1.3.2
• Python version: 3.11.2
• OS version and name: Windows 11
• pyproject.toml: see below

• [x] I am on the latest stable Poetry version, installed using a recommended method.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have consulted the FAQ and blog for any relevant entries or release notes.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Create a project as follows:

pyproject.toml:

[tool.poetry]
name = "poetry-test"
version = "0.1.0"
description = ""
authors = ["John Smith <[email protected]>"]
packages = [{include = "poetry_test"}]

[tool.poetry.dependencies]
python = "^3.11"

[tool.poetry.build]
generate-setup-file = true
script = "build.py"

[build-system]
requires = ["poetry-core", "setuptools"]
build-backend = "poetry.core.masonry.api"

build.py:

from setuptools import Extension

def build(setup_kwargs):
    setup_kwargs['ext_modules'] = [
        Extension(
            name="poetry_test.test",
            sources=["poetry_test/test.c"],
        ),
    ]

poetry_test\__init__.py: empty

poetry_test\test.c:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

PyModuleDef def = {
    PyModuleDef_HEAD_INIT,
    "poetry_test.test",
    "",
};

PyObject *PyInit_test(void) {
    return PyModule_Create(&def);
}

Now build:

> poetry build

Preparing build environment with build-system requirements poetry-core, setuptoolsBuilding poetry-test (0.1.0)
running build
running build_py
creating C:\Users\dpb\Documents\Source\test\poetry-test\build
creating C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311
creating C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311\poetry_test
copying poetry_test\__init__.py -> C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311\poetry_test
copying poetry_test\test.c -> C:\Users\dpb\Documents\Source\test\poetry-test\build\lib.win-amd64-cpython-311\poetry_test
running build_ext
building 'poetry_test.test' extension
[...snipped...]
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.752.0_x64__qbz5n2kfra8p0\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'poetry_test/test.c'
  return self._open_to_write(zinfo, force_zip64=force_zip64)
C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.752.0_x64__qbz5n2kfra8p0\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'poetry_test/__init__.py'
  return self._open_to_write(zinfo, force_zip64=force_zip64)

Note the warnings.

Now, examine the RECORD file inside the wheel:

> tar -xOf .\dist\poetry_test-0.1.0-cp311-cp311-win_amd64.whl poetry_test-0.1.0.dist-info/RECORD
poetry_test/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
poetry_test/test.c,sha256=LV2Cw22u_-WHeTIx6r4_b8QOPSxRF_46zhXfV6HkUic,207
poetry_test/test.c,sha256=LV2Cw22u_-WHeTIx6r4_b8QOPSxRF_46zhXfV6HkUic,207
poetry_test/test.cp311-win_amd64.pyd,sha256=93EsxTSbGK32w4PIDSsKumB8e0HFoW7AMfvcibYfh80,10752
poetry_test/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
poetry_test-0.1.0.dist-info/METADATA,sha256=FRvgZ3bDNIep0KQkK6KxJetccHVs-0NGTd7B8egelQE,243
poetry_test-0.1.0.dist-info/WHEEL,sha256=q-KLDJQebzIZyKLb3oMnX-cCkloRZtwcIVhfy70qOs8,98
poetry_test-0.1.0.dist-info/RECORD,,

As you can see, test.c and __init__.py are listed twice. This contradicts the spec:

The RECORD file holds the list of installed files. It is a CSV file containing one record (line) per installed file.

[SpecLad]

I believe this code is supposed to prevent this issue:

https://github.com/python-poetry/poetry-core/blob/b5d1934fed190bbfd1a303b44d5e2ef9a600efed/src/poetry/core/masonry/builders/wheel.py#L199

However, it doesn't work, because rel_path is a Windows path, while wheel.namelist() contains POSIX paths.

[radoering]

There are even warnings about this in our test environment (which I have not attached much importance to so far):

tests/masonry/test_api.py::test_build_wheel_extended
tests/masonry/builders/test_complete.py::test_wheel_c_extension
tests/masonry/builders/test_complete.py::test_wheel_c_extension_src_layout
  C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'extended/__init__.py'
    return self._open_to_write(zinfo, force_zip64=force_zip64)

tests/masonry/test_api.py::test_build_wheel_extended
tests/masonry/builders/test_complete.py::test_wheel_c_extension
tests/masonry/builders/test_complete.py::test_wheel_c_extension_src_layout
  C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\zipfile.py:1547: UserWarning: Duplicate name: 'extended/extended.c'
    return self._open_to_write(zinfo, force_zip64=force_zip64)

I assume that when fixing this bug, these tests can be extended to check that there are no duplicate entries in the RECORD file.

[chapmanjacobd]

I guess the short-term solution here is to skip building on Windows ? Not sure what else I could do

https://github.com/chapmanjacobd/library/actions/runs/4328813746/jobs/7558895451#step:6:67

[gpongelli]

this happens also with poetry 1.4 .

other OS does not suffer this issue.

[gpongelli]

as per my comment on poetry-core, that PR fixes this issue.

Thanks,

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.