facebook
Conan toolchain does not work with Conan 2, but works with Conan 1.61.0
First: the new Conan toolchain works with Conan 1.61.0.
When trying the new Conan toolchain with Conan 2, I've stumbled upon the following problems and their solutions/hacks:
conan_initdoes not generate theuser_homedirectory and does not write something into the trace file, error messages are:
Action failed: toolchains//:conan-init (conan_init)
Required outputs are missing: Action failed to produce outputs: `buck-out/v2/gen/toolchains/213ed1b7ab869379/__conan-init__/user-home`
Action failed: toolchains//:conan-init (conan_init)
Required outputs are missing: Action failed to produce outputs: `buck-out/v2/gen/toolchains/213ed1b7ab869379/__conan-init__/trace.log`
I could fix (hack around) this by adding a mkdir to prelude/toolchains/conan/conan_init.py and writing something into the tracefile:
conan_profile(args.conan, args.user_home, args.trace_file)
with open(args.trace_file, "wt", encoding="utf-8") as trace_file:
trace_file.write("HI")
homedir = Path(args.user_home)
homedir.mkdir()
- Now
conan_lockfails. Error message:
Action failed: root//:lock (conan_lock)
Local command returned non-zero exit code 1
Reproduce locally: `env -- "BUCK_SCRATCH_PATH=buck-out/v2/tmp/root/213ed1b7ab869379/__lock__/conan_lock"
/usr/bin/env "P ...<omitted>... --conanfile ./conanfile.txt --lockfile-out buck-out/v2/gen/root/213ed1b7ab869379/__lock__/conan.lock (run `buck2 log what-failed` to get the full command)`
stdout:
stderr:
ERROR: Error reading 'buck-out/v2/gen/toolchains/213ed1b7ab869379/__conan-profile-macos-arm64__/conan-profile-macos-arm64' profile: ConfigParser: Unrecognized field 'env'
Traceback (most recent call last):
File "./buck-out/v2/gen/prelude/213ed1b7ab869379/toolchains/conan/__conan_lock__/conan_lock.py", line 104, in <module>
main()
Conan does not like the [env] stanza of the generated profile file buck-out/v2/gen/toolchains/213ed1b7ab869379/__conan-profile-macos-arm64__/conan-profile-macos-arm64, as env has been deprecated in favour of buildenv
[settings]
arch=armv8
os=Macos
build_type=Release
compiler=apple-clang
compiler.version=15
compiler.libcxx=libc++
[env]
CMAKE_FIND_ROOT_PATH=
AR=$PROFILE_DIR/AR
AS=$PROFILE_DIR/AS
NM=$PROFILE_DIR/NM
RANLIB=$PROFILE_DIR/RANLIB
STRIP=$PROFILE_DIR/STRIP
CC=$PROFILE_DIR/CC
CFLAGS=
CXX=$PROFILE_DIR/CXX
CXXFLAGS=
So I changed prelude/toolchains/conan/defs.bzl to use [buildenv] instead of [env]:
content.add("")
content.add("[buildenv]")
- Now
conan_lockdoes not generate something in it's trace file. Error message:
Action failed: root//:lock (conan_lock)
Required outputs are missing: Action failed to produce outputs: `buck-out/v2/gen/root/213ed1b7ab869379/__lock__/trace.log`
So, another "HI" in prelude/toolchains/conan/conan_lock.py
conan_lock(
args.conan,
args.profile,
args.conanfile,
args.lockfile_out,
args.lockfile,
args.user_home,
args.trace_file,
)
with open(args.trace_file, "wt", encoding="utf-8") as trace_file:
trace_file.write("HI")
- Now my Conan generates a lockfile of version
0.5, butlock_generatewants0.4:
Action failed: root//:lock-generate (conan_generate)
Local command returned non-zero exit code 1
...
assert data["version"] == "0.4", "Unsupported Conan lockfile version"
And that's the end, because the Conan toolchain wants to read the field graph_lock, which does not exist any more in newer lock files.
Looks like Conan 2 stores the relevant package information in a SQLite file located at ~/.conan2/p/cache.sqlite3, the lock-file does not contain information except for hashes.
To generate a JSON file containing the same (similar) dependency graph as the Conan 1.x lockfile, the following Conan 2 command works:
conan graph info conanfile.txt --format=json > conan_deps.json
The generated JSON file looks like this:
{
"graph": {
"nodes": {
"0": {
"ref": "conanfile",
"id": "0",
"recipe": "Consumer",
"package_id": null,
"prev": null,
"rrev": null,
"rrev_timestamp": null,
"prev_timestamp": null,
"remote": null,
"binary_remote": null,
"build_id": null,
"binary": null,
"invalid_build": false,
"info_invalid": null,
"name": null,
"user": null,
"channel": null,
"url": null,
"license": null,
"author": null,
"description": null,
"homepage": null,
"build_policy": null,
"upload_policy": null,
"revision_mode": "hash",
"provides": null,
"deprecated": null,
"win_bash": null,
"win_bash_run": null,
"default_options": null,
"options_description": null,
"version": null,
"topics": null,
"package_type": "unknown",
"settings": {
"os": "Macos",
"arch": "armv8",
"compiler": "clang",
"compiler.cppstd": "gnu17",
"compiler.libcxx": "libc++",
"compiler.version": "17",
"build_type": "Release"
},
"options": {},
"options_definitions": {},
"generators": [],
"python_requires": null,
"system_requires": {},
"recipe_folder": null,
"source_folder": null,
"build_folder": null,
"generators_folder": null,
"package_folder": null,
"cpp_info": {
"root": {
"includedirs": [
"include"
],
"srcdirs": null,
"libdirs": [
"lib"
],
"resdirs": null,
"bindirs": [
"bin"
],
"builddirs": null,
"frameworkdirs": null,
"system_libs": null,
"frameworks": null,
"libs": null,
"defines": null,
"cflags": null,
"cxxflags": null,
"sharedlinkflags": null,
"exelinkflags": null,
"objects": null,
"sysroot": null,
"requires": null,
"properties": null
}
},
"conf_info": {},
"label": "conanfile.txt",
"dependencies": {
"1": {
"ref": "rapidcheck/cci.20220514",
"run": false,
"libs": true,
"skip": false,
"test": false,
"force": false,
"direct": true,
"build": false,
"transitive_headers": null,
"transitive_libs": null,
"headers": true,
"package_id_mode": null,
"visible": true
},
"2": {
"ref": "gtest/1.14.0",
"run": false,
"libs": true,
"skip": false,
"test": false,
"force": false,
"direct": true,
"build": false,
"transitive_headers": null,
"transitive_libs": null,
"headers": true,
"package_id_mode": null,
"visible": true
},
...
"7": {
"ref": "gmp/6.3.0#df20ffb6d21c34d67704305bcd1dea9e",
"id": "7",
"recipe": "Cache",
"package_id": "16d0e665c1800f360b7e6a13e496335ffb8ae715",
"prev": null,
"rrev": "df20ffb6d21c34d67704305bcd1dea9e",
"rrev_timestamp": 1698225617.026,
"prev_timestamp": null,
"remote": null,
"binary_remote": null,
"build_id": null,
"binary": "Missing",
"invalid_build": false,
"info_invalid": null,
"name": "gmp",
"user": null,
"channel": null,
"url": "https://github.com/conan-io/conan-center-index",
"license": [
"LGPL-3.0",
"GPL-2.0"
],
"author": null,
"description": "GMP is a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating-point numbers.",
"homepage": "https://gmplib.org",
"build_policy": null,
"upload_policy": null,
"revision_mode": "hash",
"provides": null,
"deprecated": null,
"win_bash": null,
"win_bash_run": null,
"default_options": {
"shared": false,
"fPIC": true,
"disable_assembly": true,
"enable_fat": false,
"run_checks": false,
"enable_cxx": true
},
"options_description": null,
"version": "6.3.0",
"topics": [
"math",
"arbitrary",
"precision",
"integer"
],
"package_type": "static-library",
"settings": {
"os": "Macos",
"arch": "armv8",
"compiler": "clang",
"compiler.cppstd": "gnu17",
"compiler.libcxx": "libc++",
"compiler.version": "17",
"build_type": "Release"
},
"options": {
"disable_assembly": "True",
"enable_cxx": "True",
"fPIC": "True",
"run_checks": "False",
"shared": "False"
},
"options_definitions": {
"shared": [
"True",
"False"
],
"fPIC": [
"True",
"False"
],
"disable_assembly": [
"True",
"False"
],
"run_checks": [
"True",
"False"
],
"enable_cxx": [
"True",
"False"
]
},
"generators": [],
"python_requires": null,
"system_requires": {},
"recipe_folder": "/Users/roland/.conan2/p/gmpa630188b79556/e",
"source_folder": null,
"build_folder": null,
"generators_folder": null,
"package_folder": null,
"cpp_info": {
"root": {
"includedirs": [
"include"
],
"srcdirs": null,
"libdirs": [
"lib"
],
"resdirs": null,
"bindirs": [
"bin"
],
"builddirs": null,
"frameworkdirs": null,
"system_libs": null,
"frameworks": null,
"libs": null,
"defines": null,
"cflags": null,
"cxxflags": null,
"sharedlinkflags": null,
"exelinkflags": null,
"objects": null,
"sysroot": null,
"requires": null,
"properties": null
}
},
"conf_info": {},
"label": "gmp/6.3.0",
"dependencies": {
"8": {
"ref": "m4/1.4.19",
"run": true,
"libs": false,
"skip": false,
"test": false,
"force": false,
"direct": true,
"build": true,
"transitive_headers": null,
"transitive_libs": null,
"headers": false,
"package_id_mode": null,
"visible": false
}
},
"context": "host",
"test": false
},
...
CC @aherrmann (with no obligation to fix). I'll raise a diff to document the restriction so at least it is clear.
@ndmitchell Thanks! Yes, indeed it was developed against Conan 1 so far. Conan 2 support would require some diving into what changed and what needs to be adjusted.
Conan 2 support would require some diving into what changed and what needs to be adjusted.
That's what I've tried to collect in the first three posts.
- dealing with no output from conan
- changing
[env]into[buildenv]in the generated profile file - using
conan graph info conanfile.txt --format=jsoninstead of the lockfile.
I would've proposed the changes that in more detail (or a PR), but I ended up using vcpkg.