`python::pip` doesn't reinstall if `extras` is updated (with no change in `ensure`)

[EBoisseauSierra]

trafficstars

Affected Puppet, Ruby, OS and module versions/distributions

• Master
  • Puppet: 6.25.0
  • Distribution: Centos 7
• Agent:
  • Puppet: 5.5.10
  • Distribution: Debian 10
  • Ruby: ruby 2.5.5p157 (2019-03-15 revision 67260) [x86_64-linux-gnu]
• Module version: d9c51a.

Problem

I can install a Python package with an arbitrary list of optional dependencies (extras) and it works just fine on a blank VM. However, if I change the list of optional dependencies and run the agent again, the new dependencies are not installed.

How to reproduce (e.g Puppet code you use)

• This works on a blank VM:

  python::pip { 'apache-superset':
    ensure       => '1.3.2',
    extras       => ['postgres'],
    virtualenv   => "${base_dir}/venv",
    pip_provider => 'pip3',
    install_args => '',
    owner        => $owner,
  }

• This works on a blank VM too:

  python::pip { 'apache-superset':
    ensure       => '1.3.2',
    extras       => ['postgres', 'mysql'],
    virtualenv   => "${base_dir}/venv",
    pip_provider => 'pip3',
    install_args => '',
    owner        => $owner,
  }

• However, this change doesn't install the optional dependency (i.e. mysqlclient>=2.1.0, <3) required by the added 'mysql' extra dependency:

 python::pip { 'apache-superset':
   ensure       => '1.3.2',
-  extras       => ['postgres'],
+  extras       => ['postgres', 'mysql'],
   virtualenv   => "${base_dir}/venv",
   pip_provider => 'pip3',
   install_args => '',
   owner        => $owner,
 }

What are you seeing

# install 'apache-superset[postgres, prophet]==1.3.2'
$ puppet agent

# (we effectively waited a few days — hence the dependencies update, see below)
# update dependencies to 'apache-superset[postgres, prophet, mysql]==1.3.2'
$ puppet agent

# manually install the package
$ pip freeze > /tmp/pip_deps_old.txt
$ pip install --upgrade --force-reinstall 'apache-superset[prophet, mysql, postgresql]'==1.3.2
$ pip freeze > /tmp/pip_deps_new.txt
$ diff /tmp/pip_deps_old.txt /tmp/pip_deps_new.txt
[…]
85c87
< msgpack==1.0.2
---
> msgpack==1.0.3
86a89
> mysqlclient==1.4.2.post1
89c92
< packaging==21.2
---
> packaging==21.3
[…]

We can see that some dependencies have been updated with more recent ones (e.g. msgpack and packaging, as the Agent hadn't run for a few days)… but more importantly, mysqlclient — required by the mysql extra dependency — is a new installed. This means that the Agent hadn't installed it.

What behaviour did you expect instead

The agent should --force-reinstall the package if the list of extra dependencies changes — even if ensure => M.m.p hasn't changed.

Any additional information you'd like to impart

In our real-life case, apache-superset has been installed via this part of our home-baked manifest.