Bug in conda version 4.5.12 leads to corrupt docker image if certain python packages are installed

[ViaFerrata]

Hi there,

the current python docker images use conda 4.5.12 as a base. However, there's a major bug in this version that leads to an error in the conda list command if you've installed certain packages via pip.

I noticed this, when I added the sphinx-autoapi package to my databricks docker image. I can replicate that installing this package leads to an error when starting a cluster with the associated docker image.

In the logs of the cluster, you can see that a conda list command is run during the startup of the cluster. And due to the bug mentioned above, the list command raises an error, such that the cluster does not start successfully.

I've upgraded the conda version in the image manually by changing the version from 4.5.12 to 4.7.12 (fixes the bug mentioned above) and the issue is fixed.

However: Is it safe to upgrade the conda in the docker image, in order to get the conda bugfix in 4.7.12, or does this lead to problems (you probably chose this particular conda version on purpose)? I couldn't find any issues till now, but of course I didn't do a proper & thorough testing.