Unifying packaging.python.org with pypa.io, possibly other PyPA docs too

[di]

We have a few web properties that represent "Python Packaging":

• https://packaging.python.org (this repo, fairly well maintained)
• https://pypa.io (https://github.com/pypa/pypa.io, infrequently updated)
• Various project-specific docs:
  • https://pip.pypa.io/
  • https://virtualenv.pypa.io/
  • https://warehouse.pypa.io/
  • etc

I'd like to propose unifying at least the first two, by rolling pypa.io into packaging.python.org, and making packaging.python.org more of a "landing page for Python packaging" (which would mostly include the existing content, but also content about what the PyPA is, brought from pypa.io)

I'd also like to move anything that's not directly related to developing PyPI and more related to using PyPI (like https://warehouse.pypa.io/api-reference/index.html) to packaging.python.org, and generally make it a policy that:

• packging.python.org is for end users
• Warehouse docs are for Warehouse contributors and maintainers

(This would be easy because the Warehouse docs are mostly for maintainers/contributors, and very little for end-users.)

This leaves other project-specific docs, like the pip docs, which are mostly for end-users, but also contain some development guides. Some possibilities:

• Nothing changes, they continue to live at *.pypa.io but pypa.io redirects to packaging.python.org
• They live at *.packaging.python.org instead
• Or, a more extreme version of this would be to (somehow?) unify all project-specific end-user documentation under packaging.python.org

I'm not sure the last one is a good idea or even technically possible, but I do think giving end-users a one-stop-shop for user guides, tutorials, etc is something worth working towards.

(cc @brainwane @pradyunsg @ewdurbin @webknjaz @pfmoore @uranusjr @gaborbernat)

[pfmoore]

I'm assuming that https://bootstrap.pypa.io would remain unchanged, as that's where get-pip.py and the virtualenv zipapp version is hosted. I imagine that is hard coded in a lot of places, so I don't think it would be easy to change.

I'm not over-keen on moving the pip docs (mostly because my muscle memory is pretty solid on pip.pypa.io 🙂) but unifying the "general" docs seems like a good plan, in principle.

[FFY00]

I am +1 on merging pypa.io into packaging.python.org. -0.5 on unifying the project documentation.

[di]

@pfmoore Thanks for noting that, I agree it shouldn't change. I also think any more extreme changes would come with appropriate redirects, so your muscle memory and our SEO would not be impaired 😉.

@FFY00 of the three options for project documentations, which would you prefer?

[FFY00]

The first.

Note that not all project docs live at *.pypi.io, I would leave it up to the projects to choose where to place them. If we want to unify, we probably want to unify everything.

[uranusjr]

I never understood or could explain why the PyPA Specifications are listed on packaging.python.org instead of pypa.io, so this is definitely a welcomed change.

I also advocate the project documentations to stay where they are.

[pradyunsg]

This might be a minority opinion, but I'm a strong -1 on all the changes proposed here.

I think it'd be a much better investment of everyone's time to better clarify what the two sites are supposed to be for; and to give some TLC to the navigation story on both those sites. And, to spend some time improving the individual projects' documentation before we move things around.

We should probably improve how pypa.io looks, and how we've structured that site, because it's... not in a great shape right now. Same for a bunch of our other tools' documentation.

Keeping the administrative things like "how PyPA works", "what is PyPA for" and stuff like that living in pypa.io is something I like. It creates a cleaner separation of "admin stuff" (pypa.io) vs "what is relevant to the user" (packaging.python.org), which we don't have any good site-navigation mechanisms right now to cleanly separate right now.

I'm not sure that a unification/moving of the various sites helps us with the potential for confusion and lack of cohesion -- the source of the issue is that we're a fairly uncoordinated group so far and our documentation hasn't gotten the TLC it deserves.

I do agree that the current separation hurts discoverability and adds some maintenance overhead, but we don't have any good navigational mechanisms in our Sphinx theme to make it easy to use anyway? I really don't understand how putting more content in packaging.python.org helps in general, because I already find it difficult to navigate and find things that I KNOW are there. The fact that no one adopted pypa-docs-theme other than pip, packaging.python.org and pypa.io (and pip's moved away too now) is significantly contributed to by the fact that it's... not that great. (yea, I have a whole rant about the PyPA Sphinx theme as it stands today, but I'll avoid going there now)

I don't think all user facing documentation for different PyPA projects should live in (or under) packaging.python.org. I think that's still a good place to have targeted "how to"s, discussions on how the broader ecosystem works, and general recommendations for workflows. The individual tooling documentation should still definitely have user facing information; because it lets us better separate "this is tool specific" vs this is interoperable and works for all of the tools.

Warehouse is possibly the only project for which moving everything to packaging.python.org makes sense, because it's a web application with only 1 instance, and how that instance works affects most other tooling in packaging.python.org. OTOH, I am not sure what kind of user docs would move to packaging.python.org vs stay on pypi.org.

[pfmoore]

This might be a minority opinion, but I'm a strong -1 on all the changes proposed here.

I'm -0 myself to be honest, so you're not a complete minority 🙂

It's not obvious to me what we'd solve with a reorganisation like this. PyPA is specifically not supposed to be in charge of individual projects, so "branding" everything together sounds to be like an extension of the PyPA charter (which would need a broader debate).

The reason I'm "only" -0 is that having a single starting point for users feels like a reasonable idea to me. But the devil is in the details, and public impressions are difficult to get right.

As a specific example, poetry isn't part of PyPA, but pipenv is - so if we brand all PyPA tools as unified under packaging.python.org, we'll risk re-opening the whole disaster around "why is pipenv the official tool, what about poetry" again. With some very careful design and wording, we might be able to avoid that, but it's hard to judge a high level proposal like this without knowing how we'd make sure of that.

I'm a strong -1 on anything that gives PyPA projects a privileged position on packaging.python.org a the expense of non-PyPA projects. And while I'm 100% sure that isn't the intention with this proposal, I don't really see how we'll avoid it happening in practice.

Hmm, thinking about it more, I'd prefer a clear split:

• pypa.io - for PyPA projects and documentation
• packaging.python.org - for Python packaging in general with an explicit policy not to prefer PyPA projects over non-PyPA ones.

Getting more information about other options like conda and poetry on packaging.python.org would help the community much more than just shuffling PyPA stuff around, IMO.

Edit: As an example of the sort of thing I mean, why do we even split https://packaging.python.org/key_projects/ between "PyPA projects" and "non-PyPA projects"? How is that relevant to the end user who just wants to understand Python packaging?

[gaborbernat]

I personally would be ok (+0.5) to merge packaging.python.org and pypa.io root. I'd keep all project documentations separate though.

[webknjaz]

I agree with what @pradyunsg and @pfmoore said. The distinction should be clear.

[brainwane]

@nlhkabu and @ei8fdb I'd like to hear your thoughts on this question, if you have time -- it feels connected to the documentation work you were doing in late 2020.