conan-center-index icon indicating copy to clipboard operation
conan-center-index copied to clipboard

Published 20 hours ago • verified publisher icon conan-io

Fix crashpad build when there is no executable named 'python' in PATH

Open petamas opened this issue 2 years ago • 25 comments

Specify library name and version: crashpad/all

crashpad fails to build if gn (the buildsystem used to build the package) fails to find an executable named python in the PATH. (Eg. if the interpreter is only available as python3.) This fix overrides gn's default behaviour of simply calling python with the interpreter used to run conan.

The error without this fix looks like this:

[6/209] ACTION //third_party/mini_chromium/mini_chromium/build:chromeos_buildflags(//third_party/mini_chromium/mini_chromium/build/config:gcc_like_toolchain)
FAILED: gen/build/chromeos_buildflags.h
python ../../third_party/mini_chromium/mini_chromium/build/write_buildflag_header.py --output build/chromeos_buildflags.h --rulename //third_party/mini_chromium/mini_chromium/build:chromeos_buildflags --gen-dir gen --definitions gen/build/chromeos_buildflags.h.flags
/bin/sh: 1: python: not found
[17/209] CXX obj/third_party/mini_chromium/mini_chromium/base/base.logging.o
ninja: build stopped: subcommand failed.
crashpad/cci.20220219:
crashpad/cci.20220219: ERROR: Package 'd4721895d5e53f886444447fc804ae4274194714' build failed
crashpad/cci.20220219: WARN: Build folder /home/petamas/.conan/data/crashpad/cci.20220219/_/_/build/d4721895d5e53f886444447fc804ae4274194714
ERROR: crashpad/cci.20220219: Error in build() method, line 202
        parallel=tools.cpu_count()), run_environment=True)
        ConanException: Error 1 while executing ninja -C out/Default client minidump crashpad_handler snapshot -j12
  • [X] I've read the guidelines for contributing.
  • [X] I've followed the PEP8 style guides for Python code in the recipes.
  • [X] I've used the latest Conan client version.
  • [X] I've tried at least one configuration locally with the conan-center hook activated.

petamas avatar Aug 12 '22 21:08 petamas

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Aug 12 '22 21:08 CLAassistant

Failure in build 1 (da97bc1940d7afb32801b6880cb0df86d88db773):

  • crashpad/cci.20210507@: Didn't run or was cancelled before finishing

  • crashpad/cci.20220219@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 12 '22 21:08 conan-center-bot

Failure in build 2 (b9b86e3a4107edf0975d1b17ba08db05c7d3e697):

  • crashpad/cci.20220219@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

  • crashpad/cci.20210507@: Didn't run or was cancelled before finishing

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 12 '22 22:08 conan-center-bot

config.yml syntax error in build 3:

Only one library can be changed in the same PR: [crashpad/all, gn/all]

conan-center-bot avatar Aug 12 '22 23:08 conan-center-bot

config.yml syntax error in build 4:

Only one library can be changed in the same PR: [crashpad/all, gn/all]

conan-center-bot avatar Aug 12 '22 23:08 conan-center-bot

Failure in build 5 (b9b86e3a4107edf0975d1b17ba08db05c7d3e697):

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

  • crashpad/cci.20210507@: CI failed to create some packages (All logs)

    Logs for packageID c8a28b2568117720dde36029eb9b89128a8fb3e3: 
    [settings]
arch=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=10
os=Linux

    [...]
[187/207] CXX obj/util/linux/util.scoped_pr_set_ptracer.o
[188/207] CXX obj/util/linux/util.thread_info.o
[189/207] CXX obj/util/linux/util.scoped_ptrace_attach.o
[190/207] ASM obj/util/misc/util.capture_context_linux.o
[191/207] CXX obj/util/linux/util.socket.o
[192/207] CXX obj/util/misc/util.time_linux.o
[193/207] CXX obj/util/misc/util.paths_linux.o
[194/207] CXX obj/util/process/util.process_memory_sanitized.o
[195/207] CXX obj/util/process/util.process_memory_linux.o
[196/207] CXX obj/util/posix/util.process_info_linux.o
[197/207] AR obj/util/libutil.a
[198/207] AR obj/snapshot/libcontext.a
[199/207] AR obj/minidump/libformat.a
[200/207] AR obj/client/libcommon.a
[201/207] AR obj/client/libclient.a
[202/207] STAMP obj/client/pthread_create.stamp
[203/207] AR obj/snapshot/libsnapshot.a
[204/207] AR obj/handler/libcommon.a
[205/207] AR obj/minidump/libminidump.a
[206/207] AR obj/handler/libhandler.a
[207/207] LINK ./crashpad_handler
crashpad/cci.20210507: Package 'c8a28b2568117720dde36029eb9b89128a8fb3e3' built
crashpad/cci.20210507: Build folder /home/conan/w/prod/BuildSingleReference/.conan/data/crashpad/cci.20210507/_/_/build/c8a28b2568117720dde36029eb9b89128a8fb3e3
crashpad/cci.20210507: Generated conaninfo.txt
crashpad/cci.20210507: Generated conanbuildinfo.txt
crashpad/cci.20210507: Generating the package
crashpad/cci.20210507: Package folder /home/conan/w/prod/BuildSingleReference/.conan/data/crashpad/cci.20210507/_/_/package/c8a28b2568117720dde36029eb9b89128a8fb3e3
crashpad/cci.20210507: Calling package()
[HOOK - conan-center.py] post_package(): [PACKAGE LICENSE (KB-H012)] OK
[HOOK - conan-center.py] post_package(): [DEFAULT PACKAGE LAYOUT (KB-H013)] OK
[HOOK - conan-center.py] post_package(): [MATCHING CONFIGURATION (KB-H014)] OK
[HOOK - conan-center.py] post_package(): [SHARED ARTIFACTS (KB-H015)] OK
[HOOK - conan-center.py] post_package(): [PC-FILES (KB-H020)] OK
[HOOK - conan-center.py] post_package(): [CMAKE-MODULES-CONFIG-FILES (KB-H016)] OK
[HOOK - conan-center.py] post_package(): [PDB FILES NOT ALLOWED (KB-H017)] OK
[HOOK - conan-center.py] post_package(): [LIBTOOL FILES PRESENCE (KB-H018)] OK
[HOOK - conan-center.py] post_package(): [MS RUNTIME FILES (KB-H021)] OK
[HOOK - conan-center.py] post_package(): [SHORT_PATHS USAGE (KB-H066)] OK
[HOOK - conan-center.py] post_package(): [MISSING SYSTEM LIBS (KB-H043)] OK
crashpad/cci.20210507 package(): Packaged 252 '.h' files
crashpad/cci.20210507 package(): Packaged 2 files: crashpad_handler, LICENSE
crashpad/cci.20210507 package(): Packaged 11 '.a' files
crashpad/cci.20210507 package(): Packaged 1 '.cmake' file: crashpad-cxx.cmake
crashpad/cci.20210507: Package 'c8a28b2568117720dde36029eb9b89128a8fb3e3' created
crashpad/cci.20210507: Created package revision 007143f5aa9af7c59a8ae4773b3d3f8a
crashpad/cci.20210507: Appending PATH environment variable: /home/conan/w/prod/BuildSingleReference/.conan/data/crashpad/cci.20210507/_/_/package/c8a28b2568117720dde36029eb9b89128a8fb3e3/bin
crashpad/cci.20210507: WARN: crashpad needs a shared libcurl library
[HOOK - conan-center.py] post_package_info(): ERROR: [CMAKE FILE NOT IN BUILD FOLDERS (KB-H019)] Exception raised from hook: list indices must be integers or slices, not str (type=TypeError) (https://github.com/conan-io/conan-center-index/blob/master/docs/error_knowledge_base.md#KB-H019) 
ERROR: 
	ConanException: [HOOK - conan-center.py] post_package_info(): list indices must be integers or slices, not str

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 12 '22 23:08 conan-center-bot

Build 5's errors all seem to be caused by this typo: https://github.com/conan-io/hooks/pull/446 or the issue fixed by this: https://github.com/conan-io/conan-center-index/pull/12190

petamas avatar Aug 12 '22 23:08 petamas

Failure in build 6 (b9b86e3a4107edf0975d1b17ba08db05c7d3e697):

  • crashpad/cci.20210507@: Didn't run or was cancelled before finishing

  • crashpad/cci.20220219@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 16 '22 08:08 conan-center-bot

@petamas Thank you for your PR! It seems like that recipe uses an old and prohibited pattern, you will need to move ConanInvalidConfiguration exception to validation method. Could you please to that hotfix? Thanks!

uilianries avatar Aug 16 '22 14:08 uilianries

Failure in build 7 (b9b86e3a4107edf0975d1b17ba08db05c7d3e697):

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 16 '22 14:08 conan-center-bot

@petamas Thank you for your PR! It seems like that recipe uses an old and prohibited pattern, you will need to move ConanInvalidConfiguration exception to validation method. Could you please to that hotfix? Thanks!

To me, it seems like it's only throws in the validate() method. I think the culprit may be one of the dependencies, gn, from my other PR: https://github.com/conan-io/conan-center-index/pull/12190

That one's a bit messed up right now, but will be fixed in a few hours.

petamas avatar Aug 16 '22 14:08 petamas

Failure in build 8 (32f1f7bf083d825d2807d592f024c7273df59446):

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 16 '22 20:08 conan-center-bot

Failure in build 9 (c3fa1e89435a9d5e7e8314c43f3df6902afca51a):

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 16 '22 20:08 conan-center-bot

Depends on https://github.com/conan-io/conan-center-index/pull/12190

uilianries avatar Aug 17 '22 11:08 uilianries

Failure in build 10 (c3fa1e89435a9d5e7e8314c43f3df6902afca51a):

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 22 '22 19:08 conan-center-bot

Seems like I'll need to rebase on top of master to get the fixes from https://github.com/conan-io/conan-center-index/pull/12190 . :(

petamas avatar Aug 22 '22 19:08 petamas

Failure in build 11 (98863382d8490082fd3e81782eb97cf871953082):

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 22 '22 19:08 conan-center-bot

Failure in build 12 (02e1ddb044c57b6788491682c8457c534afd6346):

  • crashpad/cci.20210507@: Didn't run or was cancelled before finishing

  • crashpad/cci.20220219@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 22 '22 21:08 conan-center-bot

Failure in build 12 (02e1ddb044c57b6788491682c8457c534afd6346):

  • crashpad/cci.20210507@: Didn't run or was cancelled before finishing
  • crashpad/cci.20220219@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

@uilianries: Can you help me out? I don't understand what the CI's problem is. (Also, for some reason, build number 5 was the last one that gave me a link to the actual build results, but I managed to figure out the URL pattern.)

The above error message talks about raising ConanInvalidConfiguration from the build() method, but as far as I can tell, neither crashpad nor its build system gn does that.

In the logs of build 12, specifically in the GCC 5 builds, I see this at the end:

ERROR: There are invalid packages (packages that cannot exist for this configuration):
gn/cci.20210429: Invalid ID: gn requires a compiler supporting c++17

I assume this has something to do with my problem. Previously, I thought the problem was that gn checked for C++17 compatibility in configure(), not validate(), but I fixed that issue in https://github.com/conan-io/conan-center-index/pull/12190, and apparently it didn't help. I'm not a conan expert, what could crashpad's build() method call that raises an ConanInvalidConfiguration?

petamas avatar Aug 22 '22 21:08 petamas

Failure in build 13 (d4a9f3e383fe4ec2e0d9b6d6e3ce6d3dfc4e34d7):

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 24 '22 07:08 conan-center-bot

@uilianries: Same errors even after your proposed modifications. :(

https://c3i.jfrog.io/c3i/misc/summary.html?json=https://c3i.jfrog.io/c3i/misc/logs/pr/12189/13-linux-gcc/crashpad/cci.20210507//summary.json

petamas avatar Aug 24 '22 07:08 petamas

@petamas Thank you for updating. I'll take a look. Maybe, some packages in Conan Center are outdated, based on the latest revision and we will need to regenerate them.

uilianries avatar Aug 24 '22 13:08 uilianries

Failure in build 14 (d4a9f3e383fe4ec2e0d9b6d6e3ce6d3dfc4e34d7):

  • crashpad/cci.20220219@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

  • crashpad/cci.20210507@: Didn't run or was cancelled before finishing

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Aug 31 '22 14:08 conan-center-bot

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Sep 30 '22 18:09 stale[bot]

@uilianries, do you have any idea on how we could proceed with this?

One hacky way would be to start requiring C++17 in this recipe too (as its build requirement gn does), but I'm not sure how much trouble that would cause consumers of the library. It's also possible that gn's recipe is suboptimal - eg. maybe it needs to use validate_build() instead of validate(). (I dunno, and cannot test this locally, because I don't have access to the same checkers as CI.)

petamas avatar Oct 03 '22 16:10 petamas

@petamas I did some changes which I hope work for this PR: https://github.com/petamas/conan-center-index/pull/1 (I only tested on Linux)

uilianries avatar Oct 04 '22 11:10 uilianries

Failure in build 15 (e76d977b59492dd3cc0a251983f74a995f59c10d):

  • crashpad/cci.20220219@: Error running command conan info crashpad/cci.20220219@#8480eb01af9e1585e0a5088a08f2ee36 --json {jsonName} -pr {profileName}:

    [settings]
arch=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MT
compiler.version=15
os=Windows

...
ERROR: crashpad/cci.20220219: Error in validate() method, line 81
	check_min_vs(1911)
	TypeError: check_min_vs() missing 1 required positional argument: 'version'

  • crashpad/cci.20210507@: Error running command conan info crashpad/cci.20210507@#6ade42f20efd7800afc89e00368eb5d7 --json {jsonName} -pr {profileName}:

    [settings]
arch=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MT
compiler.version=15
os=Windows

...
ERROR: crashpad/cci.20210507: Error in validate() method, line 81
	check_min_vs(1911)
	TypeError: check_min_vs() missing 1 required positional argument: 'version'

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Oct 04 '22 12:10 conan-center-bot

Failure in build 16 (b6b2c9e6a777e8ac15622118a4815e2d54f1c245):

  • crashpad/cci.20210507@: Error running command conan info crashpad/cci.20210507@#6ade42f20efd7800afc89e00368eb5d7 --json {jsonName} -pr {profileName}:

    [settings]
arch=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MT
compiler.version=15
os=Windows

...
ERROR: crashpad/cci.20210507: Error in validate() method, line 81
	check_min_vs(1911)
	TypeError: check_min_vs() missing 1 required positional argument: 'version'

  • crashpad/cci.20220219@: Error running command conan info crashpad/cci.20220219@#8480eb01af9e1585e0a5088a08f2ee36 --json {jsonName} -pr {profileName}:

    [settings]
arch=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MT
compiler.version=15
os=Windows

...
ERROR: crashpad/cci.20220219: Error in validate() method, line 81
	check_min_vs(1911)
	TypeError: check_min_vs() missing 1 required positional argument: 'version'

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Oct 04 '22 12:10 conan-center-bot

Failure in build 17 (b47eb87df2b00c5ae6be7e43736593db28af1340):

  • crashpad/cci.20210507@: Do not raise ConanInvalidConfiguration from build() method. Please, move that same check to validate() method.

  • crashpad/cci.20220219@: Didn't run or was cancelled before finishing

Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.

conan-center-bot avatar Oct 04 '22 12:10 conan-center-bot

@uilianries : Pulled in your changes, fixed some minor issues, same error as before. I tried building locally (on Linux, with gcc 5) while adding a try/except to the build() function just to see the issue happening there too, but crashpad's build() method is not called at all, it fails while pulling in gn

petamas avatar Oct 04 '22 12:10 petamas