microsoft
conda packages only work with conda-forge
I'm trying to add Playwright to my conda environment installed from Anaconda's repository, but it produces conflicts and/or tries to replace the entire environment.
It seems the packages published to anaconda.org/microsoft have tight dependencies to conda-forge, which means it's impossible to install the packages into a non-conda-forge environment.
For example, here's the output of a command that ought to install python=3.10 from Anaconda and playwright from -c microsoft:
(env) C:\Users\...> conda install -c microsoft playwright python=3.10
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package python conflicts for:
playwright -> typing_extensions -> python[version='>=2.7,<2.8.0a0|>=3.5|>=3.6|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']
playwright -> python[version='>=3.10,<3.11.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.11,<3.12.0a0|>=3.7,<3.8.0a0']
python=3.10
(As usual, I don't believe that the conflicts listed are the actual conflicts. The fact that it fails is the main point.)
If I add conda-forge as a channel, I get this proposed plan:
>conda install -c microsoft -c conda-forge playwright python=3.10
...
The following NEW packages will be INSTALLED:
bzip2 conda-forge/win-64::bzip2-1.0.8-h8ffe710_4
ca-certificates conda-forge/win-64::ca-certificates-2023.7.22-h56e8100_0
greenlet repo/main/win-64::greenlet-2.0.1-py310hd77b12b_0
libffi conda-forge/win-64::libffi-3.4.2-h8ffe710_5
libsqlite conda-forge/win-64::libsqlite-3.43.2-hcfcfb64_0
libzlib conda-forge/win-64::libzlib-1.2.13-hcfcfb64_5
openssl conda-forge/win-64::openssl-3.1.4-hcfcfb64_0
pip conda-forge/noarch::pip-23.3.1-pyhd8ed1ab_0
playwright microsoft/win-64::playwright-v1.33.0-py310_0
pyee conda-forge/noarch::pyee-9.0.4-pyhd8ed1ab_0
python conda-forge/win-64::python-3.10.13-h4de0772_0_cpython
python_abi conda-forge/win-64::python_abi-3.10-4_cp310
setuptools conda-forge/noarch::setuptools-68.2.2-pyhd8ed1ab_0
tk conda-forge/win-64::tk-8.6.13-hcfcfb64_0
typing_extensions conda-forge/noarch::typing_extensions-4.8.0-pyha770c72_0
tzdata conda-forge/noarch::tzdata-2023c-h71feb2d_0
ucrt conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
vc conda-forge/win-64::vc-14.3-h64f974e_17
vc14_runtime conda-forge/win-64::vc14_runtime-14.36.32532-hdcecf7f_17
vs2015_runtime conda-forge/win-64::vs2015_runtime-14.36.32532-h05e6639_17
wheel conda-forge/noarch::wheel-0.41.2-pyhd8ed1ab_0
xz conda-forge/win-64::xz-5.2.6-h8d14728_0
However I'm not able to use conda-forge packages in my environment - I have to use Anaconda's (for $place_of_work reasons).
It would probably be best if the anaconda.org/microsoft channel contained packages that work with Anaconda's repository, and then contribute a recipe to conda-forge to make their own build of playwright that works with that channel.
(An alternative would be to ask Anaconda to make their own build of playwright that works with their repository, but they'd want it in conda-forge first anyway, at which point the problem is solved.)
Okay, after a bit more playing, I've managed to get an environment that basically meets my needs.
The problem is the python_abi "package", which doesn't contain any files, but is used to constrain packages (basically, to make conda install fail more often so you're less likely to get a broken environment).
With the following commands, I managed to get an environment that takes python_abi from conda-forge, but has everything else from the correct locations:
> conda install python=3.10
> conda install -c conda-forge python_abi
> conda install -c microsoft playwright
> conda list
# packages in environment at C:\Users\...\envs\env:
#
# Name Version Build Channel
bzip2 1.0.8 he774522_0
ca-certificates 2023.08.22 haa95532_0
greenlet 2.0.1 py310hd77b12b_0
libffi 3.4.4 hd77b12b_0
openssl 3.0.11 h2bbff1b_2
pip 23.3 py310haa95532_0
playwright v1.33.0 py310_0 microsoft
pyee 9.0.4 py310haa95532_0
python 3.10.13 he1021f5_0
python_abi 3.10 2_cp310 conda-forge
setuptools 68.0.0 py310haa95532_0
sqlite 3.41.2 h2bbff1b_0
tk 8.6.12 h2bbff1b_0
typing_extensions 4.7.1 py310haa95532_0
tzdata 2023c h04d1e81_0
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
wheel 0.41.2 py310haa95532_0
xz 5.4.2 h8cc25b3_0
zlib 1.2.13 h8cc25b3_0
The three commands need to be run separately, otherwise they'll each try to replace most of the environment. (The conda-forge step actually downgrades my TLS certificates to an older version, but the third step brings them back.)
So there's a workaround for users. I'm not sure there's an easy way to avoid the python_abi dependency though, other than to compile using Anaconda packages instead of conda-forge. Clearly all the other dependencies are available.
I was looking at some other packages which we distribute over our microsoft conda channel, their Python dependency looks similar, and it seems like there is not much we can do here on our side? Where is our python_abi dependency coming from? Most likely from Python itself?
Seems like you are more of a Conda expert here than we are here, so if you have suggestions, let us know!
I was looking at some other packages which we distribute over our microsoft conda channel, their Python dependency looks similar
I never accused you of being the only culprits 😉
Where is our python_abi dependency coming from?
It comes from using conda-forge when building the package, because they make it a strict requirement. Anaconda does not, so if you build using their packages, it would still successfully install against conda-forge because all the dependencies are satisfied. It doesn't work the other way, because Anaconda can't satisfy the python_abi dependency.
if you have suggestions, let us know
I suspect deleting this line will be sufficient: https://github.com/microsoft/playwright-python/blob/main/.github/workflows/publish.yml#L19
Possibly it needs to be replaced with defaults, but that should be the default.
I suspect deleting this line will be sufficient: https://github.com/microsoft/playwright-python/blob/main/.github/workflows/publish.yml#L19
Possibly it needs to be replaced with defaults, but that should be the default.
I tried both, but getting this in both casese:
Unsatisfiable dependencies for platform osx-64: {'greenlet==3.0.0', 'pyee==11.0.1'}
Looks like Anaconda hasn't updated past greenlet 2.0.1 and pyee 9.0.4 for osx-x64 (list here, but be warned it's a big page).
Are those particular versions required? Or just frozen there. We can ask them to prioritise the newer versions if needed and get them added.
We usually freeze them, so that if customers install Playwright, its guaranteed that this version works for them. We had cases where dependencies were breaking us.
We are also using very often very recent versions of pyee or greenlet, since they often fix bugs which are important for our customers.
So if we would lax the version range for our published conda package, it would work with the default channel? TIL.
That sounds like a good reason, but as I said, we can ask Anaconda to accelerate their updates if there are fixes we need.
And yeah, loosening version ranges is generally a good move. Conda does a pretty good job of locking in the versions that are actually used to build a package if they matter, and allowing updates on install for those that don't. So you should be able to get the best of all worlds by not freezing the inputs and making sure that the outputs are/aren't frozen as appropriate.
Happy to take this internal to figure out how to proceed from here, you know my team alias (the obvious one).