build Build CLI stuck / pip prompt not working with private pypi index

Hi

We're trying to get a build going with a simple pyproject.toml and setuptools als the build backend. We have a pip.conf that specifies a private pypi index to be used for all installs.

Running python -m build results in the following output being stuck and not moving.

* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools>=45, wheels, setuptools_scm>=6.2)

Hitting enter twice reveals that the command was blocking the output of the pip install invocation in the isolated environment. So pip is asking for user and password using the prompt, but the prompt is hidden.

Looking in indexes: [redacted]
User for [redacted]: ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 313, in _get_abstract_dist_for
    self._populate_link(req)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 279, in _populate_link
    req.link = self.finder.find_requirement(req, upgrade)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 901, in find_requirement
    req.name, specifier=req.specifier, hashes=hashes,
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 883, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 828, in find_all_candidates
    project_url, link_evaluator=link_evaluator,
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 792, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 612, in fetch_page
    return _get_html_page(location, session=self.session)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 451, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 166, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/network/session.py", line 421, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 650, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_vendor/requests/hooks.py", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/tmp/build-env-ghmd794l/lib/python3.7/site-packages/pip/_internal/network/auth.py", line 246, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
ValueError: not enough values to unpack (expected 3, got 2)
User for [redacted]: 
Traceback (most recent call last):
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 373, in main
    args.srcdir, outdir, distributions, config_settings, not args.no_isolation, args.skip_dependency_check
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 229, in build_package_via_sdist
    sdist = _build(isolation, builder, outdir, 'sdist', config_settings, skip_dependency_check)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 140, in _build
    return _build_in_isolated_env(builder, outdir, distribution, config_settings)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/__main__.py", line 108, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/env.py", line 211, in install
    _subprocess(cmd)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/env.py", line 81, in _subprocess
    raise e
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/site-packages/build/env.py", line 78, in _subprocess
    subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/home/stefan/.pyenv/versions/3.7.12/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/tmp/build-env-ghmd794l/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-t6halb98.txt']' returned non-zero exit status 2.

ERROR Command '['/tmp/build-env-ghmd794l/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-t6halb98.txt']' returned non-zero exit status 2.

Which leads to two questions:

Would it be possible to display the prompt normally, so the user can manually enter the credentials to an actual promp? Because entering myusername<Enter>mypassword<Enter> seems to work just fine, but one has to figure that out first. Also the user is prompted multiple times, leading to multiple "hidden" prompts.

Is there a way to "inject" the pip call somehow to provide the credentials in a non-interactive way? For example in the CI, we use HTTP basic auth in the form of pip install -i https://username:[email protected]/simple. Another way could be to somehow pre-configure keyring to be installed and initialized. Worst case we could probably use .netrc, but at least for local machines we would like to avoid this (see question 1 regarding multiple prompts.)

On a side note, I'm a bit puzzled that this didn't come up in #246 .

Nov 24 '21 17:11 sbor23

Thanks for reporting, we should be passing --no-input to pip. How does passing --no-input to pip change the experience in your use case?

Nov 24 '21 18:11 FFY00

As expected, this leads the the same issue as if the remote was not known, so:

* Creating venv isolated environment...
* Installing packages in isolated environment... (setuptools>=45, setuptools_scm>=6.2, wheel)
Looking in indexes: [redacted]
ERROR: Could not find a version that satisfies the requirement wheel (from versions: none)
ERROR: No matching distribution found for wheel

Traceback (most recent call last):
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 372, in main
    built = build_call(
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 229, in build_package_via_sdist
    sdist = _build(isolation, builder, outdir, 'sdist', config_settings, skip_dependency_check)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 140, in _build
    return _build_in_isolated_env(builder, outdir, distribution, config_settings)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/__main__.py", line 108, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/env.py", line 212, in install
    _subprocess(cmd)
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/env.py", line 81, in _subprocess
    raise e
  File "/home/stefan/projects/pyorgan/venv/lib/python3.10/site-packages/build/env.py", line 78, in _subprocess
    subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/usr/lib64/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib64/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/build-env-jb75xsz0/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '--no-input', '-r', '/tmp/build-reqs-ejsueipu.txt']' returned non-zero exit status 1.

ERROR Command '['/tmp/build-env-jb75xsz0/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '--no-input', '-r', '/tmp/build-reqs-ejsueipu.txt']' returned non-zero exit status 1.

Nov 25 '21 14:11 sbor23

I guess this is related to: https://github.com/pypa/pip/issues/10665

However, from my experience pip will use the keyring for credentials if available, while build doesn't.

Jan 24 '22 13:01 sbor23

For anybody looking for a workaround, set the environment variable PIP_NO_INPUT=1 before running build.