puppet-python

puppet-python copied to clipboard

Pull Request (PR) description

For Ubuntu 18.04 and Debian 10, ensure distutils is installed.

When testing pyvenv I ran into this problem when using Python 3 and noticed if you install python3-venv instead of python3.7-venv it pulls in python3-distutils but rather than changing the behavior of how to install python*-venv just pulling in the dependency works.

Failure with/without system packages

root@debian10:/# python3.7 -m venv --clear --system-site-packages /opt/globus-cli
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/opt/globus-cli/bin/python3.7', '-Im', 'ensurepip', '--upgrade', '--default-pip']

root@debian10:/# python3.7 -m venv --clear /opt/globus-cli
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/opt/globus-cli/bin/python3.7', '-Im', 'ensurepip', '--upgrade', '--default-pip']

Solution:

root@debian10:/# apt-get install python3.7-distutils
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'python3-distutils' instead of 'python3.7-distutils'
The following packages were automatically installed and are no longer required:
  libexpat1-dev libpython-all-dev libpython-dev libpython2-dev libpython2.7 libpython2.7-dev python-all
  python2-dev python2.7-dev
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  python3-distutils
0 upgraded, 1 newly installed, 0 to remove and 14 not upgraded.
Need to get 142 kB of archives.
After this operation, 715 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 python3-distutils all 3.7.3-1 [142 kB]
Fetched 142 kB in 0s (333 kB/s)       
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package python3-distutils.
(Reading database ... 42675 files and directories currently installed.)
Preparing to unpack .../python3-distutils_3.7.3-1_all.deb ...
Unpacking python3-distutils (3.7.3-1) ...
Setting up python3-distutils (3.7.3-1) ...

root@debian10:/# python3.7 -m venv --clear --system-site-packages /opt/globus-cli
root@debian10:/# 

Seems installing python3.7-distutils actually installs python3-distutils and causes issues with not being idempotent.

@treydock thanks for the PR! Are you able to add an acceptance test here as well?

@bastelfreak Any suggestions on what kind of acceptance tests to add? There are already numerous tests for this defined type that the catalog applies successfully. The only thing I could think to test is that python3-distutils is actually installed for these select operating systems, since the there are already tests for success of setting up the pyvenv environment.

if I read this correctly, the code on master fails if you set the python version on debian 10 to 3.7? Is that something that can be validated with a test?

I am fairly certain for at least Debian 10, which is one of the OSes where I encountered this, the Python 3 version installed is 3.7 so I just do something like this in my module:

  class { 'python':
    version => '3',
  }
  python::pyvenv { 'globus-cli':
    ensure     => 'present',
    version    => 'system',
    venv_dir   => '/opt/globus-cli',
    systempkgs => true,
  }

That's when I encountered the error for several Debian based operating systems. I was trying to switch my Globus module to Python 3 for all operating systems: https://github.com/treydock/puppet-module-globus/pull/22

That is why I'm not sure why the existing acceptance tests didn't catch this because the code I'm using looks very similar to what's already being done. I think after looking more closely I see why the tests didn't catch this. If I set python::dev: present then everything works so I'm guessing when you install the python dev packages on Debian it pulls in the necessary packages.

So this PR is not necessary if the dev => 'present' is set for python class. I wonder if this is just something that needs to be documented rather than adding this edge case?

I wonder if maybe a warning is the appropriate change. If python::dev is absent, issue a warning in pyvenv defined type?

Dear @treydock, thanks for the PR!

This is Vox Pupuli Tasks, your friendly Vox Pupuli GitHub Bot. I noticed that your pull request contains merge conflict. Can you please rebase?

You can find my sourcecode at voxpupuli/vox-pupuli-tasks