Create a new guide for setting up a baseline Python environment

[ncoghlan]

A couple of things I've realised we need to clearly establish for the packaging user guide are:

• Which commands can folks contributing to the packaging guide or working on the PyPI user experience assume will "just work" on an end user's machine?
• How much guidance should the packaging tutorials provide on ensuring a system is in that state, given the wide variety of potential starting states?

Right now, we don't clearly document those environmental assumptions anywhere, which then makes it hard to provide a checklist for people to work through and ensure their environment is set up correctly.

The closest we currently get to this is to include a fairly cursory introduction to virtualenv in the package installation tutorial, and then implicitly assuming that the following is true in the rest of the guide:

• the user is working at a command prompt (whether inside their IDE, or in an actual system shell)
• the command python will run the desired version of Python
• the commands pip install and python -m pip install will be essentially equivalent (aside from the differences when upgrading pip itself on Windows) and install packages into the desired version of Python
• any executables installed that way will be executable from that command prompt
• any Python import packages installed that way will be importable from a Python shell started in that environment

Those assumptions are true once you're inside an activated virtual environment, but they're not true in the general case due to the differences between per-user Python installations and system-wide ones, and the differences in cross-platform conventions when it comes to learning how to start Python.

(Issue prompted by https://mail.python.org/pipermail/python-ideas/2017-November/047708.html, #394, and https://github.com/pypa/pip/issues/3164)

Additional related documentation:

• Bootstrapping guide for Linux: https://packaging.python.org/guides/installing-using-linux-tools/
• CPython's getting started guide: https://docs.python.org/3/installing/

[ncoghlan]

Something else this would let us do is document how various ways of obtaining Python fare in terms of meeting the baseline assumptions:

• platform provided *nix Python 3.x installations
• python.org Mac OS X installers
• python.org Windows installers (both user installs and system installs)

We'd then encourage alternate distributors to document how their particular solution covers the baseline assumptions.

[ncoghlan]

Something I'm wondering is whether or not we could reduce the assumptions to an actual concrete set of commands for a user to optionally work through as a troubleshooting script, such as:

# Checking python & pip run *and* match
$ python --version
Python 3.6.2
$ python -c "import sysconfig; print(sysconfig.get_path('purelib'))"
/home/ncoghlan/.virtualenvs/cpython/lib/python3.6/site-packages
$ pip --version
pip 9.0.1 from /home/ncoghlan/.virtualenvs/cpython/lib/python3.6/site-packages (python 3.6)

# Checking command installation
$ pip install pipsi
... installer output ...
$ pipsi --version
pipsi, version 0.9, python /home/ncoghlan/.virtualenvs/cpython/bin/python3

# Checking library installation
$ pip install requests
... installer output ...
$ python -c "import requests; print(requests)"
<module 'requests' from '/home/ncoghlan/.virtualenvs/cpython/lib/python3.6/site-packages/requests/__init__.py'>

And then have a "Python installation trouble-shooting guide" that walked through each command and the various ways they can go wrong, like:

>>> python --version
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'python' is not defined

[ncoghlan]

While they're not the main focus of the PR, I've made some improvements to this in https://github.com/pypa/python-packaging-user-guide/pull/402

[theacodes]

There's a lot of good ideas here, but I'm unsure of what is actionable. @ncoghlan can you help me break this down into concrete things that need to happen? Do we need a new guide called Setting up a baseline Python environment or similar?

[ncoghlan]

Yeah, I think a new Guide would be a good way to handle this.

#399 is an example of needing this kind of information, since "apt-get install python3" will give you a Python where venv's are still missing pip by default.

[pradyunsg]

Is anyone working on this?

This would be really useful and I'm happy to help in any way to push this forward. Happy to write as well, if someone would be willing to provide some guidance. :)

[ncoghlan]

I'm not aware of anyone currently working on it, and even a relatively rough first version becomes something we can iterate on based on feedback.

[theacodes]

Go for it. I'm always happy to review. You're welcome to just put up a skeleton/outline and I can review that to ensure structure is good before you start fleshing out sections.

[pradyunsg]

Awesome. Will definitely post a structure in a bit.

I'm gonna do a lot of presentations + writing + planning work this weekend. This should fit right in. :)

[pradyunsg]

Just noting here that I didn't get the time to work on this.

[ncoghlan]

From https://github.com/pypa/pipenv/issues/2122, another potential issue that can come up is situations where PATH and sys.path may end up checking different locations in different orders, such that a command found on PATH doesn't find the matching version of itself on sys.path.

[pradyunsg]

Coming to this now, I'm thinking of roughly something like:

• Introduction Talk about why we have this document
• Expectations from the environment Have a list here, of what things should be available and things that should match up etc
  • Verification Provide instructions on how to verify that your environment matches the requirements above
  • Troubleshooting If an environment isn't meeting the baseline requirements, some pointers on how to debug them.
• Platform Specific Instructions How do you get from a "fresh" system to an environment described above. (this is where we link to the existing tutorials/guides)
  • Cross Platform ways?
  • Linux
  • Windows
  • MacOS

[pradyunsg]

And that last section also provides any platform specific instructions needed. (like you need to not fiddle with system packages on Linux and Mac, PATH caveats, not needing a virtualenv on Windows etc)

[pradyunsg]

Thinking about the expectations:

• executing in a command line
• have a way to invoke a supported python interpreter
• have working pip on that interpreter
• able to install packages and scripts
• scripts installed are visible (first) on PATH when installed via pip

[pradyunsg]

That last comment went prematurely. Edited to completion.

Thoughts?

[ncoghlan]

Yep, I think that's what we have in mind.

[ncoghlan]

Via @mariatta, there's a new Real Python article covering getting set up across different platforms: https://realpython.com/installing-python/

[pradyunsg]

Thinking about the "validate your Python setup" automation, I'm facing a sort of bootstrapping issue. There's no cross-platform way to actually run something like a script or automated checking unless we're first able to assume that the user knows how to run a script using an installed Python interpreter.

One idea was to explain clearly to the user how to run a script with Python and then use those instructions to make it possible for them to run the script. Any pointers on what we can do for this?

---

Unrelated: Should we also check pythonX and the corresponding pipX versions in the automated check?