os icon indicating copy to clipboard operation
os copied to clipboard

airflow/2.10.0 package update

Open octo-sts[bot] opened this issue 1 year ago • 4 comments

octo-sts[bot] avatar Aug 16 '24 02:08 octo-sts[bot]

Open AI suggestions to solve the build error:

The error message is: 

fatal: detected dubious ownership in repository at '/github/home' To add an exception for this directory, call:

git config --global --add safe.directory /github/home ERRO request failed error="Get "./packages/apk-configuration": unsupported protocol scheme """ method=GET url=./packages/apk-configuration ERRO failed to build package: unable to build guest: unable to generate image: installing apk packages: installing packages: installing mariadb-11.4-dev (ver:11.4.3-r0 arch:aarch64): unable to install files for pkg mariadb-11.4-dev: unable to install file over existing one, different contents: usr/include/mysql/mariadb_rpl.h make[1]: *** [Makefile:111: packages/aarch64/airflow-2.10.0-r0.apk] Error 1 make[1]: Leaving directory '/github/home' make: *** [Makefile:101: package/airflow] Error 2


Steps to fix:
1. Run `git config --global --add safe.directory /github/home`.
2. Ensure the URL in the GET request uses a supported protocol.
3. Resolve the file conflict in `mariadb_rpl.h`.
4. Re-run the `make` command.

octo-sts[bot] avatar Aug 16 '24 02:08 octo-sts[bot]

Open AI suggestions to solve the build error:

The error message is: "fatal: detected dubious ownership in repository at '/github/home'
To add an exception for this directory, call:

git config --global --add safe.directory /github/home
ERRO request failed error=\"Get \\\"./packages/apk-configuration\\\": unsupported protocol scheme \\\"\\\"\" method=GET url=./packages/apk-configuration
ERRO failed to build package: unable to build guest: unable to generate image: installing apk packages: installing packages: installing mariadb-11.4-dev (ver:11.4.3-r1 arch:aarch64): unable to install files for pkg mariadb-11.4-dev: unable to install file over existing one, different contents: usr/include/mysql/mariadb_rpl.h
make[1]: *** [Makefile:111: packages/aarch64/airflow-2.10.0-r0.apk] Error 1
make[1]: Leaving directory '/github/home'
make: *** [Makefile:101: package/airflow] Error 2
##[error]Process completed with exit code 2."

1. Run `git config --global --add safe.directory /github/home`.
2. Correct the URL in the GET request to include a valid protocol.
3. Ensure the correct version of `mariadb_rpl.h` is used or remove the existing file before installation.
4. Re-run the make command.

octo-sts[bot] avatar Aug 19 '24 16:08 octo-sts[bot]

Open AI suggestions to solve the build error:

The error message is: "failed to build package: unable to build guest: unable to generate image: installing apk packages: installing packages: installing mariadb-11.4-dev (ver:11.4.3-r1 arch:aarch64): unable to install files for pkg mariadb-11.4-dev: unable to install file over existing one, different contents: usr/include/mysql/mariadb_rpl.h"

1. Remove the conflicting file: `rm /usr/include/mysql/mariadb_rpl.h`
2. Clean the package manager cache: `apk cache clean`
3. Update the package list: `apk update`
4. Retry installing the package: `apk add mariadb-11.4-dev`

octo-sts[bot] avatar Aug 19 '24 18:08 octo-sts[bot]

Open AI suggestions to solve the build error:

The error message is: "fatal: detected dubious ownership in repository at '/github/home'
To add an exception for this directory, call:

git config --global --add safe.directory /github/home
ERRO request failed error=\"Get \\\"./packages/apk-configuration\\\": unsupported protocol scheme \\\"\\\"\" method=GET url=./packages/apk-configuration
ERRO ERROR: failed to build package. the build environment has been preserved:
INFO   workspace dir: /temp/melange-workspace-3549208363
INFO   guest dir: /temp/melange-guest-1497087693
ERRO failed to build package: unable to build guest: unable to generate image: installing apk packages: installing packages: installing mariadb-11.5-dev (ver:11.5.2-r1 arch:aarch64): unable to install files for pkg mariadb-11.5-dev: unable to install file over existing one, different contents: usr/include/mysql/mariadb_rpl.h
make[1]: *** [Makefile:111: packages/aarch64/airflow-2.10.0-r0.apk] Error 1
make[1]: Leaving directory '/github/home'
make: *** [Makefile:101: package/airflow] Error 2
##[error]Process completed with exit code 2."

1. Run `git config --global --add safe.directory /github/home`.
2. Ensure the URL in the GET request uses a supported protocol (e.g., http or https).
3. Resolve the file conflict for `usr/include/mysql/mariadb_rpl.h`.
4. Re-run the build process.

octo-sts[bot] avatar Aug 22 '24 18:08 octo-sts[bot]

Open AI suggestions to solve the build error:

The error message is: "fatal: detected dubious ownership in repository at '/github/home'
To add an exception for this directory, call:

git config --global --add safe.directory /github/home
ERRO request failed error=\"Get \\\"./packages/apk-configuration\\\": unsupported protocol scheme \\\"\\\"\" method=GET url=./packages/apk-configuration
INFO Hunk #1 FAILED at 478.
INFO Hunk #2 succeeded at 520 (offset 19 lines).
INFO 1 out of 2 hunks FAILED -- saving rejects to file hatch_build.py.rej
ERRO ERROR: failed to build package. the build environment has been preserved:
INFO workspace dir: /temp/melange-workspace-2001934074
INFO guest dir: /temp/melange-guest-1811236875
ERRO failed to build package: unable to run package airflow pipeline: unable to run pipeline: unable to run pipeline: exit status 1
make[1]: *** [Makefile:111: packages/aarch64/airflow-2.10.0-r0.apk] Error 1
make[1]: Leaving directory '/github/home'
make: *** [Makefile:101: package/airflow] Error 2
##[error]Process completed with exit code 2."

1. Run `git config --global --add safe.directory /github/home`.
2. Ensure the URL in the GET request has a valid protocol (e.g., `http://` or `https://`).
3. Fix the patch for `hatch_build.py` to ensure all hunks apply correctly.
4. Investigate logs in `/temp/melange-workspace-2001934074` and `/temp/melange-guest-1811236875`.
5. Ensure all dependencies and configurations for the `airflow` package are correct.
6. Re-run the build process.

octo-sts[bot] avatar Aug 25 '24 00:08 octo-sts[bot]

Open AI suggestions to solve the build error:

The error message is: "ERROR: Cannot install apache-airflow==2.10.0 and connexion because these package versions have conflicting dependencies.\nERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts\nERROR: failed to build package. the build environment has been preserved:\nfailed to build package: unable to run package airflow pipeline: unable to run pipeline: exit status 1\nmake[1]: *** [Makefile:111: packages/aarch64/airflow-2.10.0-r0.apk] Error 1\nmake[1]: Leaving directory '/github/home'\nmake: *** [Makefile:101: package/airflow] Error 2\n##[error]Process completed with exit code 2."

1. Check dependency requirements for `apache-airflow==2.10.0` and `connexion`.
2. Identify conflicting dependencies.
3. Update `requirements.txt` or `setup.py` with compatible versions.
4. Run `pip install` again.
5. Use a virtual environment or Docker if conflicts persist.

octo-sts[bot] avatar Sep 02 '24 13:09 octo-sts[bot]

Given:

  • https://github.com/spec-first/connexion/pull/1585

I don't understand how upstream managed to release this.

xnox avatar Sep 02 '24 15:09 xnox

Open AI suggestions to solve the build error:

No errors were found in the log file.

octo-sts[bot] avatar Sep 04 '24 19:09 octo-sts[bot]

# apk info -W /opt/airflow/venv/lib/python3.12/site-packages/werkzeug-2.3.8.dist-info/METADATA
/opt/airflow/venv/lib/python3.12/site-packages/werkzeug-2.3.8.dist-info/METADATA is owned by airflow-2.9.3-r2

Currently we ship airflow 2.9.3 with werkzeug 2.3.8 - is it thus broken / dead-on-arrival? As the above upgrade, claims that airflow is not compatible with werkzeug 2.3.x series, and thus previous mediation, actually just broke the package? Do we need to undo advisory, say that it is actually broken, and then upgrade with a known CVE, pedning upstream fix (i.e. airflow needs to figure out how to support 2.3.x) or werkzeug needs to publish 2.2.3.1 with CVE fix (and all the advisory metadata needs to be updated).

xnox avatar Sep 04 '24 21:09 xnox

Can we instead remove connections functionality?

I've checked for it but it seems it used in lots of parts, i don't think it's possible to drop this dep.

and/or remove werkzeug functionality?

It seems werkzeug also required dep, since it used to create password hashes and handle some HTTP exceptions. README also mentions "werkzeug tightly coupled with Flask libraries"

do we need provide a public backport of the 2.3.8 fix into 2.2.x series of werkzeug?

It'd be great, they don't support older versions. The related CVE fix commit does seem: https://github.com/pallets/werkzeug/commit/f2300208d5e2a5076cbbb4c2aad71096fd040ef9

Dentrax avatar Sep 04 '24 21:09 Dentrax

Thanks @Dentrax, @xnox, @jamie-albert. Seems like we shouldn't have bumped werkzeug originally as it does like its not compatible, and the above links reaffirm this. Lets leave it at the supported version, and update our advisory to pending-upstream-fix, and we'll keep track for any changes upstream

mamccorm avatar Sep 05 '24 01:09 mamccorm