Python build fails with --intermediate-dir and setuptools_scm

[carderne]

Bug Report

At what date and time did you most recently experience the problem?

Now.

Where did you experience the problem? E.g. Azure Web Apps, Azure Functions, Azure Container Registry, or offline use.

Running the AzureRmWebAppDeployment@4 Azure pipeline task on a Python app. The command it uses is:

/bin/bash -c "oryx build /tmp/zipdeploy/extracted \
  -o /home/site/wwwroot \
  --platform python \
  --platform-version 3.8 \
  -i /tmp/blablabla \
  --compress-destination-dir \
  -p virtualenv_name=antenv \
  --log-file /tmp/build-debug.log  | tee /tmp/oryx-build.log ; exit $PIPESTATUS "

The full pipeline config is:

- task: AzureRmWebAppDeployment@4
  inputs:
    ConnectionType: 'AzureRM'
    azureSubscription: 'text-here'
    appType: 'webAppLinux'
    WebAppName: 'app-name-here'
    packageForLinux: 'path-to-zip-file'
    RuntimeStack: 'PYTHON|3.8'
    StartupCommand: 'gunicorn ...'
    AppSettings: '-SCM_DO_BUILD_DURING_DEPLOYMENT true'

What happened?

We're using setuptools_scm for versioning, which needs access access to the .git directory to get version information. But that dir is stripped out here: https://github.com/microsoft/Oryx/blob/e805ef4a86e4edb65a4bd0bf4f5a851a8c26c562/src/BuildScriptGenerator/DefaultBuildScriptGenerator.cs#L145

I've tried manually setting COMMAND in the AppSettings as follows, but it is ignored by the builder.

    AppSettings: '-SCM_DO_BUILD_DURING_DEPLOYMENT true
                  -COMMAND "oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --compress-destination-dir -p virtualenv_name=antenv"

What did you expect or want to happen?

The package gets built with version info from setuptools_scm.

How can we reproduce it?

1. git init
2. echo . > requirements.txt
3. Place the following in pyproject.toml:

[build-system]
requires = ["setuptools>=65", "wheel>=0.37.1", "setuptools-scm[toml]>=7.0.5"]
build-backend = "setuptools.build_meta"
[project]
name = "foo"
requires-python = ">=3.8"
dynamic = ["version"]
[tool.setuptools_scm]

4. oryx build ./ -o /tmp/foo -i /tmp/bar

Do you have log output? Please include between the backticks:

LookupError: setuptools-scm was unable to detect version for /tmp/pip-req-build-me6l4jjp.
Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj

[carderne]

TLDR: Build fails using Oryx from a Web app Linux deployment when using setuptools_scm because it uses --intermediate-dir.

Possible solutions:

1. Pass a different Onyx command to the pipeline and exclude --intermediate-dir flag (is this possible?).
2. Don't exclude .git/ from the intermediate dir (know idea what consequences this might have).
3. Don't use a versioner that relies on .git access (what I'm currently doing).

[william-msft]

Hi @carderne, I notice --platform-version 3.8 \ could you try using 3.10?

[carderne]

The problem is that the .git folder is excluded from the --intermediate-dir. This will be the same regardless of the Python version.

Just tried with 3.9 and got the same error. Tried 3.10 and got errors on the Python installation script that GLIBC_2.xx wasn't found (running the mcr.microsoft.com/oryx/build:latest docker image).

[simonjj]

We're closing this issue due to its age. If this is still impacting you please open a new issue and simply link back to this one.