setuptools_scm breaks pip installing subdirs from git

[jck]

For example: pip install 'git+https://github.com/swift-nav/libsbp#subdirectory=python'

pip clones the entire repo, including the .git dir, but setuptools_scm breaks

Collecting git+https://github.com/swift-nav/libsbp#subdirectory=python
  Cloning https://github.com/swift-nav/libsbp to /tmp/jck/pip-pjPlzH-build
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/jck/pip-pjPlzH-build/python/setup.py", line 55, in <module>
        zip_safe=False)
      File "/usr/lib64/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/jck/.local/share/virtualenvs/a990cbf97408406/lib/python2.7/site-packages/setuptools/dist.py", line 323, in __init__
        _Distribution.__init__(self, attrs)
      File "/usr/lib64/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/home/jck/.local/share/virtualenvs/a990cbf97408406/lib/python2.7/site-packages/setuptools/dist.py", line 392, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/jck/pip-pjPlzH-build/python/.eggs/setuptools_scm-1.15.6-py2.7.egg/setuptools_scm/integration.py", line 22, in version_keyword
      File "/tmp/jck/pip-pjPlzH-build/python/.eggs/setuptools_scm-1.15.6-py2.7.egg/setuptools_scm/__init__.py", line 119, in get_version
      File "/tmp/jck/pip-pjPlzH-build/python/.eggs/setuptools_scm-1.15.6-py2.7.egg/setuptools_scm/__init__.py", line 97, in _do_parse
    LookupError: setuptools-scm was unable to detect version for '/tmp/jck/pip-pjPlzH-build/python'.
    
    Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
    
    For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/jck/pip-pjPlzH-build/python

[RonnyPfannschmidt]

@dstufft i'm not sure if setuptools_scm is at fault or if pip breaks us - any idea?

[dstufft]

We're probably only copying the sub directory and not the parent directory.

[RonnyPfannschmidt]

@dstufft does pip provide any mechanism to get pkg-info/egg-info in place even in such a case

[RonnyPfannschmidt]

for the record, we do already have fallback support for pip-egg-info that works for normal checkouts

it doesn't work for sub-directory checkouts

[dstufft]

If I'm right above, probably not? Though it might be reasoanble to copy the parent directory too and just CD into the sub directory-- but we'd have to verify that is actually the case.

[pitrou]

The issue seems not only with git subdir URLs but also when you have a complete checkout and run pip install . in a subdirectory. See original report for PyArrow at https://issues.apache.org/jira/browse/ARROW-2067

(as a sidenote, it would be nice to explain somewhere why this package is "blessed"? how is it better than e.g. versioneer?)

[ikus060]

I have the same issue here. In my setup.py:

use_scm_version={"root": "..", "relative_to": __file__}

Running python setup.py --version after git clone is working properly. Trying to install using pip with subdirectory is failling. Little digging with the /tmp directory reveal it's cause by the creation of pip-egg-info directory. When the directory exists, it fail like this:

looking for ep setuptools_scm.parse_scm_fallback .
found ep pip-egg-info = setuptools_scm.hacks:parse_pip_egg_info
root '/tmp/pip-InXmsP-build/minarca-plugins'
looking for ep setuptools_scm.parse_scm /tmp/pip-InXmsP-build/minarca-plugins
looking for ep setuptools_scm.parse_scm_fallback /tmp/pip-InXmsP-build/minarca-plugins
found ep pip-egg-info = setuptools_scm.hacks:parse_pip_egg_info
pip-egg-info /tmp/pip-InXmsP-build/minarca-plugins/pip-egg-info []
Traceback (most recent call last):
  File "setup.py", line 175, in <module>
    'MinarcaServerInfo = minarca_server_info',
  File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 372, in __init__
    _Distribution.__init__(self, attrs)
  File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
    self.finalize_options()
  File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 528, in finalize_options
    ep.load()(self, ep.name, value)
  File "/usr/local/lib/python2.7/dist-packages/setuptools_scm/integration.py", line 22, in version_keyword
    dist.metadata.version = get_version(**value)
  File "/usr/local/lib/python2.7/dist-packages/setuptools_scm/__init__.py", line 119, in get_version
    parsed_version = _do_parse(root, parse)
  File "/usr/local/lib/python2.7/dist-packages/setuptools_scm/__init__.py", line 97, in _do_parse
    "use git+https://github.com/user/proj.git#egg=proj" % root)
LookupError: setuptools-scm was unable to detect version for '/tmp/pip-InXmsP-build/minarca-plugins'.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git c
heckout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=pr
oj

If the directory exists, root parameter pass to use_scm_version is discarded and the version is not resolved. Issue is around integration.py:

    # this piece of code is a hack to counter the mistake in root finding
    matching_fallbacks = iter_matching_entrypoints(
        '.', 'setuptools_scm.parse_scm_fallback')
    if any(matching_fallbacks):
        value.pop('root', None)

Since a fallback is found, the pop the root arguments. Not sure to understand the logic. As soon the directory 'pip-egg-info' exists, it's "matching". The entry point is not execute. To work around this issue, I downgrade the version to 1.5.0 as follow:

setup_requires=["setuptools_scm==1.5.0"],

[RonnyPfannschmidt]

thanks for the investigation - im going to experiment with alternatives to root pop soon

[rpink01]

@RonnyPfannschmidt Any progress on a fix for this regression?

[dm0]

It is still possible to workaround this issue without downgrading setuptools_scm. It works for me with the following setup (setup.py is one level deeper than repository root)

...
use_scm_version={'relative_to': os.path.dirname(__file__)},
...

Please let me know if this solution has any drawbacks.

[abhin-rk]

Any progress on this?

[RonnyPfannschmidt]

modern pip no longer breaks this as far as i can tell

[xkortex]

Hi, I think I'm having a similar issue to this. Here's a repo where I've been playing around with the utility.

https://github.com/xkortex/test_scm/tree/2252ab3b53b3cda31fdd82ebc90d3c113bd8d987

This is with pip 20.1.1. I have tried dozens of configurations, messing with root, relative_to etc and I cannot seem to get it to pip install when a setup.py is present.

I'll try to play around with different configurations when I have more time. I'm pretty sure I'm following the documentation, but it's not totally clear when pip517 is in play or not.

[johnandersen777]

Same here