Shebang line is sometimes not rewritten because it is #!/bin/sh

[oseiberts11]

We have a package called s11stack-manager which is built for Ubuntu 16.04LTS Xenial and 18.04LTS Bionic on Launchpad and installs a virtualenv. In the installed files, the scripts in the bin directory (such as bin/pip) refer to a non-existing python:

#!/bin/sh
'''exec' /build/s11stack-manager-3u9Lm2/s11stack-manager-0.18.1+syseleven+bionic/debian/s11stack-manager/opt/syseleven/venvs/s11stack-manager/bin/python3 "$0" "$@"
' '''

The build process leaves a reference to the build directory /build/s11stack-manager-3u9Lm2/s11stack-manager-0.18.1+syseleven+bionic/debian/s11stack-manager/opt/syseleven/venvs/s11stack-manager/bin/python instead of correcting it to /opt/syseleven/venvs/s11stack-manager/bin/python. This seems to be a bug in dh_virtualenv. It has code to changes the paths, but it expects those paths to be in the first line of the scripts, following #!. However, sometimes, virtualenv creates a different script beginning, like so:

#!/bin/sh
'''exec' /home/syseleven/s11stack-manager-0.18.1+syseleven+bionic/debian/s11stack-manager/opt/syseleven/venvs/s11stack-manager/bin/python3 "$0" "$@"
' '''

and that pattern is not recognised, so it is not fixed. (that is from a manual invocation on a Bionic installation)

See code: /usr/lib/python2.7/dist-packages/dh_virtualenv/deployment.py fix_shebangs(self).

This bug is also visible when I invoke the dh_virtualenv command myself on Ubuntu 16.04 and 18.04. There is apparently no package of dh_virtualenv for Ubuntu 20.04LTS; this matches with what's listed on https://packages.ubuntu.com/search?keywords=dh-virtualenv .

Our debian/rules file is

#!/usr/bin/make -f
export DH_VIRTUALENV_INSTALL_ROOT=/opt/syseleven/venvs
export S11STACK_MANAGER_CONFIG=

%:
        dh $@ --with python-virtualenv

override_dh_virtualenv:
        dh_virtualenv --extra-virtualenv-arg "--never-download" --extra-pip-arg "--no-index" --extra-pip-arg "--find-links=file://${PWD}/debian/pypi" --extra-pip-arg "--no-cache-dir" --python /usr/bin/python3 --builtin-venv --requirements requirements-builddeb.txt

[oseiberts11]

I tried to report a link to this bug at Launchpad (for Ubuntu packaging), where https://launchpad.net/dh-virtualenv seems to be the place. See https://bugs.launchpad.net/ubuntu/+source/dh-virtualenv/+bug/1905425 .

[nailor]

Thanks for the report!

I don’t follow launchpad bugs, since primarily I only publish this to Debian unstable. But thanks for the report anyhow, does not hurt with extra visibility 😊

I’m guessing what we are seeing is Python packaging tools being smart and detecting too long shebang lines, replacing them with the trick above.

In order to consider this being fixed would need to verify that is indeed the case and of course do the replace in a way we don’t accidentally cause any unwanted side effects, albeit that’s probably not a big worry.

Definitely up for accepting fixes for this, given we look into the two things mentioned above. In the meantime, you could probably do the magic yourself by overwriting a suitable step after dh_virtualenv has ran in your debian/rules file

On Tue 24. Nov 2020 at 17.21, O Seibert Syseleven [email protected] wrote:

I tried to report a link to this bug at Launchpad (for Ubuntu packaging), where https://launchpad.net/dh-virtualenv seems to be the place. See https://bugs.launchpad.net/ubuntu/+source/dh-virtualenv/+bug/1905425 .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/spotify/dh-virtualenv/issues/316#issuecomment-733040089, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAPRSEJIEOLEJYKK52B5TLSRPE6HANCNFSM4UA37WNQ .

--

• Jyrki

[strugee]

This appears to be a regression: #225

There are tests for it and everything.

[strugee]

@oseiberts11 were you installing dh_virtualenv from APT on bionic and xenial? If so, you're running a package that did not include the fix for #225, in which case this bug should probably be closed as already fixed.

[oseiberts11]

@strugee I think the answer to that is "yes". As far as I can see from Launchpad build logs, they do use APT. When I tried it locally, I'm pretty sure I did the same. So if this is fixed in later versions, that's fine with me! In the mean time, we have put a workaround in our build process.

[strugee]

@oseiberts11 yes, Launchpad uses APT. Sorry, I should have been more specific with my question - I was wondering if you were installing dh_virtualenv from your distribution's repositories (i.e. Ubuntu universe) or somewhere else. Can you copy and paste the Launchpad configuration you have in /etc/apt/sources.list.d? Or alternately, the command you use to add Launchpad as a source in your build system? (Presumably an invocation of add-apt-repository, but maybe something else.)