shebang for tool.poetry.scripts gets too long for scripts created for venv

[jylind]

• Poetry version: 1.2.2
• Python version: 3.8.10
• OS version and name: Ubuntu 18.04

• [x] I am on the latest stable Poetry version, installed using a recommended method.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have consulted the FAQ and blog for any relevant entries or release notes.

Issue

Script files created for entries [tool.poetry.scripts] might contain shebang lines which are longer than ~128 bytes which will fail when executed.

Following will try to explain the situation: In folder /var/jenkins_home/workspace/taf_dev/v4/ci/atc-dryrun/profiles_and_permanence venv is created and the full path to venv bin folder becomes: /var/jenkins_home/workspace/taf_dev/v4/ci/atc-dryrun/profiles_and_permanence/.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/

Now the problematic script content is:

#!/var/jenkins_home/workspace/taf_dev/v4/ci/atc-dryrun/profiles_and_permanence/.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/python
import sys
from tasystem.wsb.wsbcta import main

if __name__ == '__main__':
    sys.exit(main())

Running the file will fail:

.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/system-wsb: line 2: import: command not found
.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/system-wsb: line 3: from: command not found
.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/system-wsb: line 6: syntax error near unexpected token `main'
.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/system-wsb: line 6: `    sys.exit(main())'

System tries to run that python content with bash (default in the system).

It seem that running above in Ubuntu 22.04 does not produce the same problem (not fully tested though).

But it seems that Poetry does have some kind of "protection" for this because all the other scripts created from dependencies are created differently. Bellow is an example of script created for one of the dependencies.

#!/bin/sh
'''exec' /var/jenkins_home/workspace/taf_dev/v4/ci/atc-dryrun/profiles_and_permanence/.poetry/venvs/ta-system-JbxFE9SW-py3.8/bin/python "$0" "$@"
' '''
# -*- coding: utf-8 -*-
import re
import sys
from cta.app.wsb.wsb import cli
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(cli())

Running that has no problems.

One more observation. If the shebang line will get under the limit (what ever that is) all created script files will have the form of

#!/var/jenkins_home/workspace/taf_dev/v4/ci/atc-dryrun/p_n_p/.poetry/venvs/ta-system-g0t6XxPn-py3.8/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from cta.app.wsb.wsb import cli
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(cli())

i.e. they will shebang pointing for the venv python rather than default shell.

So the problem is that scripts created for "local" project (in local pyproject.toml file) are not fixed for long shebang line the same way as scripts created from project dependencies.

[neersighted]

I don't have time to look at this in depth yet, but I first suggest that you configure virtualenvs.in-project which will result in much shorter paths without other changes to your existing layout.

We may be able to use a relative path here, or to reuse the strategy that pip/setuptools have. It does look promising -- thanks for that.

[jylind]

Thanks for the response. In this particular case I have many ways of work around this problem. My Jenkins job requires multiple tools here which all create their own venv. I would rather like to keep all of them in one place, root of the workspace, thus, not using in-project option.

Just reported this problem as I think that project scripts and scripts created from dependencies should be handled the same way (at least if both originates from poetry projects). It took me fair amount of time to figure out the reason my job was failing and hope other don't need to if this gets fixed.

[m-gris]

Hi :) Complete noob's question: How should one specify the shebang path if virtualenvs.in-project is set to true ? Thx in advance for your help.