distutils icon indicating copy to clipboard operation
distutils copied to clipboard

Init file in stubs-only package

Open gleb-chipiga opened this issue 4 years ago • 20 comments

Project structure:

aiojobs-stubs
└── aiojobs-stubs
    └── __init__.pyi
└── aiojobs_protocols
    └── __init__.py
    └── py.typed
└── LICENSE
└── MANIFEST.in
└── setup.py

setup.py

setup(
    ...
    package_data={'aiojobs-stubs': ['__init__.pyi'],
                  'aiojobs_protocols': ['py.typed']},
    packages=['aiojobs-stubs', 'aiojobs_protocols'],
    python_requires='>=3.8'
)

MANIFEST.in

include LICENSE

Run command:

python -m build

Receive message:

package init file 'aiojobs-stubs/__init__.py' not found (or not a regular file)

According to PEP 561, the __init__.py file is not needed in such a package.

gleb-chipiga avatar Jan 12 '22 10:01 gleb-chipiga

~~Folder name should be aiojobs_stubs not aiojobs-stubs (must be importable). The outer folder name can be anything, but the inner one must be importable.~~

(I was corrected later, but to clarify this is wrong, it is intentionally unimportable at runtime)

henryiii avatar Jan 12 '22 17:01 henryiii

aiojobs-stubs is correct per PEP 561. This message is from setuptools (or distutils) and AFAICT it doesn't cause a failure. I suppose setuptools expects you to declare it as a namespace package because it lacks an __init__.py. You might want to ask on their tracker.

layday avatar Jan 12 '22 17:01 layday

@abravalheri might know :)

layday avatar Jan 12 '22 17:01 layday

Hi @gleb-chipiga, I confirm that I see the warning, but everything builds fine (so hopefully you can carry on your work without much trouble).

These are the commands I used to replicate:

% cd /tmp
% mkdir -p test_aiojobs_stubs
% cd test_aiojobs_stubs
% mkdir aiojobs-stubs aiojobs_protocols
% touch aiojobs-stubs/__init__.pyi aiojobs_protocols/{__init__.py,py.typed} LICENSE
% echo 'include LICENSE' > MANIFEST.in
% echo "from setuptools import setup
setup(
name='aiojobs-stubs',
package_data={'aiojobs-stubs': ['__init__.pyi'],
              'aiojobs_protocols': ['py.typed']},
packages=['aiojobs-stubs', 'aiojobs_protocols'],
python_requires='>=3.8'
)" > setup.py
% virtualenv .venv -p py38
% .venv/bin/python -m pip install -U pip build
% .venv/bin/python -m build
% unzip -l dist/aiojobs_stubs-0.0.0-py3-none-any.whl
Archive:  dist/aiojobs_stubs-0.0.0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2022-01-12 18:19   aiojobs-stubs/__init__.pyi
        0  2022-01-12 18:19   aiojobs_protocols/__init__.py
        0  2022-01-12 18:19   aiojobs_protocols/py.typed
        0  2022-01-12 18:28   aiojobs_stubs-0.0.0.dist-info/LICENSE
      183  2022-01-12 18:28   aiojobs_stubs-0.0.0.dist-info/METADATA
       92  2022-01-12 18:28   aiojobs_stubs-0.0.0.dist-info/WHEEL
       32  2022-01-12 18:28   aiojobs_stubs-0.0.0.dist-info/top_level.txt
      655  2022-01-12 18:28   aiojobs_stubs-0.0.0.dist-info/RECORD
---------                     -------
      962                     8 files

This is caused by the implementation of pypa/distutils that predates PEP 561. (Ideally setuptools should also include *.pyi automatically).

This issue could be reported both in pypa/setuptools (FR for automatically adding *.pyi) and pypa/distutils (remove the warn if all the files inside the folder are *.pyi).

abravalheri avatar Jan 12 '22 18:01 abravalheri

Ahh, I hadn't thought about that before, but it makes sense.

Shouldn't pypa/distutils or pypa/setuptools also check for __init__.pyi and avoid the warning for that case too?

Edit: Actually, that's what you suggest at the end, yes.

henryiii avatar Jan 12 '22 18:01 henryiii

Should this issue be moved? I can move it to packaging-problems, where it might get more views, or someone else could move it to distutils or setuptools.

henryiii avatar Jan 12 '22 18:01 henryiii

I think GitHub only allow maintainers of both the "origin" and the "destination" repository to move the issue 😅.

If you think packaging-problems is the most appropriate place, maybe that is a good solution (I honestly don't know what is the best approach).

abravalheri avatar Jan 12 '22 18:01 abravalheri

I think GitHub only allow maintainers of both the "origin" and the "destination"

Yes, that's why I'm limited to moving it to packaging-problems. :) I think new issues could be opened for distutils (adding pyi to the check) and setuptools (adding .pyi files, and py.typed files to the default files) would be a good idea. I'm not sure what the policy / ideal situation is for changing distutils, due to it still existing in the stdlib too.

henryiii avatar Jan 12 '22 18:01 henryiii

@jaraco is able to move it.

layday avatar Jan 12 '22 18:01 layday

setup.py gives no such warning. I only get it when using build.

from setuptools import setup

Run command:

setup.py sdist bdist_wheel

gleb-chipiga avatar Jan 12 '22 20:01 gleb-chipiga

I don't think the setup.py behavior is very important, though it is interesting to note:

https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

henryiii avatar Jan 12 '22 20:01 henryiii

The pip wheel . behavior would be interesting, though.

henryiii avatar Jan 12 '22 20:01 henryiii

I can reproduce it with setup.py so I'm not sure why you can't.

layday avatar Jan 12 '22 20:01 layday

pip wheel . worked without warning.

gleb-chipiga avatar Jan 12 '22 20:01 gleb-chipiga

Python 3.9.9

build==0.7.0 setuptools==60.5.0 pip==21.3.1 wheel==0.37.1

gleb-chipiga avatar Jan 12 '22 20:01 gleb-chipiga

(venv) ~/D/t/aiojobs-stubs (main|…) $ python -V
Python 3.9.9
(venv) ~/D/t/aiojobs-stubs (main|…) $ python -m pip list
Package    Version
---------- -------
pip        21.3.1
setuptools 60.5.0
wheel      0.37.1
(venv) ~/D/t/aiojobs-stubs (main|…) $ python setup.py sdist bdist_wheel | grep package
package init file 'aiojobs-stubs/__init__.py' not found (or not a regular file)

layday avatar Jan 12 '22 20:01 layday

There is indeed a warning. I didn't notice it. Sorry.

gleb-chipiga avatar Jan 12 '22 20:01 gleb-chipiga

I'm not sure what the policy / ideal situation is for changing distutils, due to it still existing in the stdlib too.

I only skimmed the issue, so did not see this. But distutils in ths stdlib should not receive any updates, the development has been moved to this repo, which is periodically synced with the local setuptools copy, and will eventually be merged there.

FFY00 avatar Jan 13 '22 00:01 FFY00

Folder name should be aiojobs_stubs not aiojobs-stubs (must be importable). The outer folder name can be anything, but the inner one must be importable.

@henryiii I wanted to clarify that this is incorrect, one of the goals of the -stubs suffix is to make the folder non-importable, it exists solely for static analysis purposes.

(I mention this since someone came across your comment and was confused, so you may want to edit it and either clarify or point to this comment)

emmatyping avatar Nov 03 '22 19:11 emmatyping

OK, I fixed it, though you only have to scroll down a couple of comments to see that it was an incorrect statement.

henryiii avatar Nov 03 '22 21:11 henryiii