Audit and improve error messages when building `python_distribution`

[kaos]

I may be a bit dense (but probably not more so than the average user) and struggled for a while to figure out why I suddenly got NoOwnerError: No python_distribution target found to own <source file>. errors when running ./pants package.

It took me too long to realise that the source in question where not imported by any other code in the distribution, so was effectively not a part of it. It was not until there were code from another distribution that attempted to import that code, that the error showed up (made me look in all the wrong places).

Granted, that code shouldn't be floating around, but all things are not yet in place... also, it's gratifying to see, that it wasn't pulled into any distribution when the only dependency came from another dist.

I've been thinking on how to ease troubleshooting this, and the only thing that came to mind was to issue a warning earlier, about orphan python_library when there is a "parent" distribution one might suspect it is intended to be a part of, well before any outside dependencies fail because of it.

[Eric-Arellano]

Thanks for opening this! Generally, the setup-py implementation is indeed pretty tricky to support multiple distributions in the same repo. I agree with you that the error messages could be improved to better explain the nuance of everything.

I expanded the scope of this issue to look at python_distribution error messages overall. Improving error messages are a huge priority for the project as the community grows.

[jsirois]

If its not clear, this is an instance of Pants trying to take away a foot gun. From that much complexity ensues. The foot gun is not allowing you to mistakenly publish the same module in more than one distribution. Although we never observed this problem happening in Python over the lifetime of Pants (since ~2010), this protection was added for Python very early on - in perhaps late 2011. Publishing the same classfile in more than one jar was the motivator and a frequent and huge source of real world pain in the jvm / maven ecosystem.

[kaos]

Yes, agreed. I do see what it is complaining about, and why. My issue was little in the other direction than the focus of the documentation and also your comment, John, is talking about. As in my case, it was not about to be published twice, but not at all, so without this safety hatch, the application would fail at runtime when trying to import modules that didn't exist.

So I think the check is (more than) fine, I'm just hoping we can help shine a light on which foot one was about to shoot, so to speak ;)

Apart from a added warning about orphan libraries, for this particular case at least (maybe it will never happen again.. and the effort would be wasted) is that it could suggest how to resolve the issue, by adding a explicit dependency to the (closest) distribution for the library.

This is a learning curve issue. As I get more familiar with pants and how everything ties together, I will know what to do with errors like these, so I'd say these improvements would be for relatively new pants users (or for me on one of those days.. that just doesn't go my way :D)

[github-actions[bot]]

This issue has been open for over one year without activity and is not labeled as a bug. It has been labeled as stale to invite any further updates. If you can confirm whether the issue is still applicable to the latest version of Pants, appears in other contexts, or its priority has changed, please let us know. Please feel free to close this issue if it is no longer relevant.