Can Packages be installed "editable" like "pip install -e ..."?

[guettli]

Please tell us in the docs, if packages can be installed "editable" like "pip install -e ..".

Thank you.

I could not find the information here: http://clarete.li/curdling/search.html?q=editable&check_keywords=yes&area=default

[clarete]

Hi, thanks for reporting the issue.

Curdling doesn't actually support this feature. Documentation wise, I ended up following the other way around. I only documented the most stable features. I didn't even mention a few things I'm working on because they're not stable enough.

I don't really use the pip -e option, so I'd love to understand a little bit more about the workflow of people that use it. I read about it but I guess field experience here might count a little bit more here :)

[clarete]

Not sure but I guess someone else asked something related to this matter in another issue. If that's not the case, feel free to rename this issue with something like "support editable packages".

[guettli]

We use the "-e" feature in our company.

In the requirements.txt file, most code that is from our company gets installed with

-e git+https://source/repos/foo@release/0.2.12#egg=foo

This makes it easy to set up new development enviroments.

I think it is a nice feature. Please ask, if you want to know more details.

[clarete]

So, curdling does provide a few git features, like specifying a given commit to be downloaded. It is indeed documented but the experience in curdling is really bad. If you try to install a URL without using git+ it might just compromise your cache! Sorry about that.

[EvaSDK]

The editable source feature comes in handy when working on a package that must depend on unreleased features of another packages.

The syntax for this for pip would be:

$ pip install -e git+ssh://host.tld/projects/A.git@branch_or_commit#egg=pkgname-version

and A requirements.txt would include a line like:

-e git+ssh://host.tld/projects/B.git@branch_or_commit#egg=pkgname-version

Running pip install this way would generate a working environment without extra steps. Generally speaking, version would be dev or something like this.

I see that curd does not support this -e switch in requirements.txt which makes it incompatible with projects doing it this way.

Then there is there are projects not using requirements.txt for this but setup.py and dependency_links. This is a bit more complicated but I guess you have all the logic already.

Again project A depends on unreleased project B. A's setup.py would contain something like:

from setuptools import setup

setup(
    name='A',
    install_requires=['B >=1, ==dev']
    dependency_links=[
        'git+ssh://host.tld/projects/B.git@branch_or_commit#egg=B-dev',
    ]
)

This feature was moved behind the --process-dependency-links switch because the feature extends to more than using it this way like using private indexes etc, but for what concerns me, this is the best use of dependency_links.

Finally coming back to the editable source feature, a common practice when working on a project that does not have a requirements.txt is to clone it, create a virtualenv and run:

$ pip install -e .

this will go through setup.py to install the dependencies and create relevent egg* to make cloned code available to the venv. Walking through dependency_links if asked. This allows modifying the code and running it immediately.

I hope I am clear enough in bringing some context to this ticket, this is a complex subject and might require hands on experience to properly understand but if you have any question don't hesitate.

[EvaSDK]

Also, the >=1, ==dev trickery has to do with how dependency_links, pip and setuptools play together.

pip looks for best version matching constraint. If it can't find anything, it will lookup dependency_links for a URL for the requested module and will find a matching entry for version dev which luckily is listed in version constraint. It then proceeds to installing from this source. Then (I guess) setuptools checks versions in the constraint string against module version.

[msabramo]

I would also love support for editable packages (the -e of pip install).

I want to see if I can use curdling to spin up the creation of a a dev environment that involves doing a good 20+ pip installs and this is the slowest part of the provisioning, so if I could speed it up with curdling, that would be awesome!

I need to be able to replace this pip command:

    /opt/webapp/{{ sm_app_role }}/bin/pip install
      -e /opt/src/{{ sm_app_role }}
      -i {{ pypackage_repo }}
      --find-links=/opt/pip/wheels --find-links=/opt/pip/cache

In a similar vein, I'd love to be able to do:

• curd install /opt/src/profilesvc (this directory contains a setup.py)
• curd install . (when run inside a directory with a setup.py)

Cc: @aconrad, @sudarkoff