Include pip requirements

[Raphtor]

Is support to include minimal requirements of pip planned?

For example (with conda-minify installed on the base env):

> conda env create -n test_env
> conda activate test_env
> conda install pip
...
> pip install numpy
...
> conda env export
name: test_env
channels:
  - conda-forge
dependencies:
  - ca-certificates=2021.5.30=h4653dfc_0
  - certifi=2021.5.30=py39h2804cbe_0
  - ncurses=6.2=h9aa5885_4
  - openssl=1.1.1k=h27ca646_0
  - pip=21.2.1=pyhd8ed1ab_0
  - python=3.9.6=h54d631c_1_cpython
  - python_abi=3.9=2_cp39
  - readline=8.1=hedafd6a_0
  - setuptools=49.6.0=py39h2804cbe_3
  - sqlite=3.36.0=h72a2b83_0
  - tk=8.6.10=hf7e6567_1
  - tzdata=2021a=he74cb21_1
  - wheel=0.36.2=pyhd3deb0d_0
  - xz=5.2.5=h642e427_1
  - zlib=1.2.11=h31e879b_1009
  - pip:
    - numpy==1.21.1
> conda deactivate
> conda-minify --name test_env
name: test_env
channels:
- conda-forge
dependencies:
- pip=21.2.1

My expectation is that numpy would be included under the dependencies, but this is not the case!

I think this could be done by utilizing conda env export instead of conda list in environment.py.

[jamespreed]

The support for PIP installed packages is not really rolled out yet. I can look into it, especially with the recent changes to PIP

[Serhiy-Shekhovtsov]

It's weird but for some reason conda-minify does export pip deps on Windows but not on Ubuntu. My solution is to combine the tool with pip-chill. The script looks like this:

# export pip packages
pip-chill > requirements.txt

# export minimal conda environment.yml
conda run -n base conda-minify --name my_env_name -f environment.yml -i python

# on some systems conda-minify will not export pip dependencies
if grep -L -- "- pip:" environment.yml; then
    # adding pip deps to conda env yml
    echo "- pip:\n  - -r requirements.txt" >> environment.yml
fi

Please note that conda-minify is called from base env. Calling it from within the target env crashes.

[rcasero]

@Serhiy-Shekhovtsov

echo "- pip:\n - -r requirements.txt" >> environment.yml

I think this line should be

echo -e "- pip:\n - -r requirements.txt" >> environment.yml

to avoid a literal \n instead of a newline.

[indigoviolet]

@Serhiy-Shekhovtsov pip-chill is nice! But does it not include conda-installed packages in your requirements.txt? Do you do anything about the duplication?

@jamespreed conda-minify is great! would be even more awesome if it would consider pip requirements as well!

[Serhiy-Shekhovtsov]

But does it not include conda-installed packages in your requirements.txt? Do you do anything about the duplication?

Don't remember this. I have switched to pipenv for all my projects and pretty happy so far.

[jamespreed]

I am so sorry for neglecting this project. My time has been sucked dry as my family has grown. I am hoping to put some effort into it soon.