Includes do not seem to work correctly

[tallero]

Description

Hello, I'm opening this issue just for conscience as if I really needed to fix these includes I would do it in the downstream distribution recipe script but anyway.

I can't seem to make the include/exclude fields work for a cythonized module using the reported syntax from the documentation.

This is the pyproject.toml file I've been using.

This is the build.log, this is poetry.config.log.

Workarounds

Delete the extra files from the downstream distribution build recipe.

Poetry Installation Method

system package manager (eg: dnf, apt etc.)

Operating System

Life / DogeOS

Poetry Version

1.8.4

Poetry Configuration

It's attached above.

Python Sysconfig

No response

Example pyproject.toml

I've linked it in the description above.

Poetry Runtime Logs

I'm adding it as an attachment above because it seems too long for the github issue.

[dimbleby]

You are using the wrong build-backend, please see the docs

[tallero]

I've switched back to setuptools and ditched poetry and pep517 entirely, funny @dimbleby. Also by doing so not only I have no include/exclude issues but building the package requires 2 minutes versus one hour on a desktop computer with this mess. On Android it does not even manage to finish the build as the virtual environment, which apparently can't be disabled, pretends to re-build half of the system in a single run and requires over 1GB of storage. And I am the Android poetry packager, be aware of that. You people had me lose more than a month of work and slowed down build time by a 10x factor on the whole python stack, and still everything needs to be rebuilt at every new python release. If this thing is poetry, then I'm Dante Alighieri.

[dimbleby]

If you are here for help then please use the right build backend - and make clear what happened vs what you expected to happen. All you have said so far is that it is something to do with include/exclude: but we have no way of knowing what you think poetry is actually doing wrong.

If you are here only to vent then: job done, now please close the issue.

[tallero]

I gave you the sources and the logs. You have all you need to reproduce the issue and know what poetry is doing wrong sir.

If you are here only to show how you are not here to fix the bugs, job done, now close the issue.

[dimbleby]

please say what happened vs what you expected to happen.

[tallero]

Fourth line

I can't seem to make the include/exclude fields work

Ninth line

(workaround is to) Delete the extra files from the downstream distribution build recipe.

Sixth line

here is the code you can use to reproduce the issue.

You really challenge one's good faith you are just not here to mock people, sir.

[Secrus]

Please tone down @tallero, @dimbleby

[tallero]

@Secrus im sorry but certain people seem to easily mistake community sunday nights bug reports for help requests.

[dimbleby]

you have forgotten to tell us what the bug is!

All that the logs show is that poetry successfully builds a wheel.

Presumably there is something that you were expecting either to be included or excluded from that wheel, and for which the opposite is the case. But you haven't said what, and it is hard to guess.

In good faith: please tell us what happened vs what you expected to happen.

[Secrus]

@tallero

1. As dimbleby pointed out, you are using the wrong build backend (you define poetry >=0.12 and poetry.masonry.api, while proper values should be poetry-core >= 1.x (x being whatever version you want to use) and poetry.core.masonry.api).
2. include/exclude system was refactored and is available on the main branch of poetry-core here, updated docs are available here. Could you check if it solves your issue?
3. While poetry doesn't support disabling the build isolation, you can use build, which has that option, and still use poetry-core as the build backend.

[tallero]

@dimbleby why have you even start reading the log without having looked at the pyproject.toml? Imagine if that was empty! Of course there is something I expected to included/excluded differently, and it's literally the only notable thing in that file.

@Secrus So the issue is I was using the refactored >=1.x include format with a backend from the 0.1x times? More in general I'm really puzzled as to how PEP517/PEP660 has solved anything in regard to package building, as since the switch setuptools requires build to build and build requires setuptools to build, so in the end packagers still require an external Python distribution to bootstrap core packages and still version-independent modules are installed into version-specific locations like /usr/lib/python<version>/site-packages, so requiring a new re-build each X months, so that in the end the greatest maintainable python libraries seem to be those which install themselves into their own locations at /usr/lib/<program> and depend on external build systems like GNU Make, meson and the likes.

So anyway I've edited the pyproject.toml file to use poetry.core.masonry.api instead of poetry.masonry.api but still the pyx files are included in both the source distribution and the wheel. Also if I try to run python -m build --verbose --wheel --no-isolation in the directory there is no output at all.

Again, here is the pyproject.toml, here is the master branch targeting build recipe.

The log is being produced right now, but really I struggle to see at all how poetry may be fit at all for system packaging, since despite the various flags I have switched I can't seem to make it use the locally installed (and verified) dependencies instead of (unsecurely) retrieving and rebuilding them from pypi, given it could make any sense to use a virtual environment for producing system packages.

[tallero]

Ok, so now it seems at least the .c files have been excluded from the wheel, but still the .pyx files are there. Here is the new build.log.

But really until it keeps trying to rebuild half of the system (and it even hides the relative logs unless it fails) on Android (so on a very resource-constrained environment), I can't see how I can even think of enabling it as the default build system.

[dimbleby]

but still the pyx files are included in both the source distribution and the wheel.

I infer that this is what you find unexpected. I wish you would have said so sooner!

It looks to me as though https://github.com/python-poetry/poetry-core/blob/4749d63c822147b0dbaa82033c02628f92c97200/src/poetry/core/masonry/builders/builder.py#L94 expects exclude to be a plain list of strings, a dictionary of form {"path": "this", "format": "that"} simply is not supported.

It's not clear to me whether that's intended or not. The docs "clarification" at #8852 implied that it was expected to work - but perhaps it never was.

[dimbleby]

... which makes this a duplicate of #3380, see in particular https://github.com/python-poetry/poetry/issues/3380#issuecomment-731273686

edit: though I guess that the docs have recently become more misleading is new

[tallero]

@dimbleby the statement "exclude defaults to both sdist and wheel" induced me to think it was possible to select the destination as for the includes, as indeed one may want to exclude the .pyxs only from the wheel.

[dimbleby]

right, that's exactly the text that was added in #8852, I agree that it is misleading

I expect that a pull request correcting it to say that exclude always applies to both formats would be welcome

[tallero]

@dimbleby I think it should be made so excludes apply to a destination as well. Also it should be mentioned they are both ignored by the subsequent call to setup.py when generate-setup-files = true is set in [tool.poetry.build]. I mean, right now I have no idea how to write so that a file is only included in the source distribution.

[dimbleby]

sure, I expect that a pull request to make exclude more like include would be welcome too. But

• it'll be more work, #3380 has been open for some time now and no-one has yet found it worth their while
  • (if you do - great!)
• it's already tracked by #3380, there's no need to keep this issue open to describe that enhancement

[tallero]

@dimbleby I would be glad to fix that issue, but right now I don't think I have the time to as I have to find a way to save myself from starvation before the end of the month as regardless of how important are the issues one solves for the community, it doesn't currently provide contributors any way to survive by doing so, and as a consequence for now I think I will stick to still use setuptools.

If you're interested to this particular issue I'm talking about, I suggest to take a look at my work on the matter of providing free software developers and package maintainers with a way to be paid for their work by forcing end users (not user developers) to acquire free (as in freedom) software through a smart contract on a blockchain, even though experience tells me people are just gonna keep acquiring whatever massive advertising by monopolistic actors and the subsequent peer pressure tell them to.