[feature] `conan lock` should support package lists

[andrey-zherikov]

What is your suggestion?

Reading through CI tutorial I noticed that conan expects users to call conan lock create for every package for every configuration. This becomes unmaintainable very easy when one has multiple configurations and more than a dozen of packages. Hence the feature request to support package lists so I'd need iterate over configurations only:

conan lock create --list <pkglist>
conan lock create --list <pkglist> -s build_type=Debug

Have you read the CONTRIBUTING guide?

• [x] I've read the CONTRIBUTING guide

[memsharded]

Hi @andrey-zherikov

Thanks for your suggestion.

Reading through CI tutorial I noticed that conan expects users to call conan lock create for every package for every configuration.

This is not the expectation. The expectation is for the organization "products". There could be hundreds of packages in an organization, but the final products could be just one or two packages.

Hence the feature request to support package lists so I'd need iterate over configurations only:

The problem is a lockfile only works as a "snapshot" of an actual dependency graph. It is not possible to take an arbitrary package list and make it a lockfile, because it is very likely that it wouldn't be an instance of an actual "realizable" dependency graph. Furthermore, it is not possible to apply configurations to package lists, only to dependency graphs. Trying to expand the dependency graph of a package list for a given configuration will easily lead to conflicts, because there are no guarantees that such package list is the result of a dependency graph.

[andrey-zherikov]

Reading through CI tutorial I noticed that conan expects users to call conan lock create for every package for every configuration.

This is not the expectation. The expectation is for the organization "products". There could be hundreds of packages in an organization, but the final products could be just one or two packages.

If "product" is a "deployable software" then we have dozens of products and hundreds of packages. One of the options I'm considering is to make a "meta-product" package that contains nothing but depends of everything I need. And when I add/remove product, I don't need to patch CI - just that meta package (which is much safer).

The problem is a lockfile only works as a "snapshot" of an actual dependency graph. It is not possible to take an arbitrary package list and make it a lockfile, because it is very likely that it wouldn't be an instance of an actual "realizable" dependency graph. Furthermore, it is not possible to apply configurations to package lists, only to dependency graphs. Trying to expand the dependency graph of a package list for a given configuration will easily lead to conflicts, because there are no guarantees that such package list is the result of a dependency graph.

I'm not sure I got you right. IMHO any package list is a dependency graph, however the latter might consist of multiple isolated subgraphs which is fine.

The request here is to simplify this:

for pkg in $pkg_list; do conan lock create --requires=$pkg ...; done

and just call this instead:

conan lock create --list=pkg_list ...

So if conan lock create --requires is not able to produce correct lock file nor conan lock create --list should be.

[memsharded]

I'm not sure I got you right. IMHO any package list is a dependency graph, however the latter might consist of multiple isolated subgraphs which is fine.

conan list zlib/*:* --format=json -r conancenter > pkglist.json

It is impossible that such pkglist.json is the result of a dependency graph.

[memsharded]

for pkg in $pkg_list; do conan lock create --requires=$pkg ...; done

This is doable without iteration with:

conan lock create --requires=zlib/1.3.1 --requires="bzip2/[*]" --requires=...

[andrey-zherikov]

I'm not sure I got you right. IMHO any package list is a dependency graph, however the latter might consist of multiple isolated subgraphs which is fine.

conan list zlib/*:* --format=json -r conancenter > pkglist.json

It is impossible that such pkglist.json is the result of a dependency graph.

Well, we use different term definition here but I agree with you that this package list is not valid to produce lock file.

However why can't conan lock create --list pkglist.json fail the same way as this one?

>conan lock create --requires=zlib/1.2.11 --requires=zlib/1.2.12 --requires=zlib/1.2.13 --requires=zlib/1.3 --requires=zlib/1.3.1
ERROR: Duplicated requirement: zlib/1.2.12

Also note that this:

conan lock create --requires=zlib/1.2.11
conan lock create --requires=zlib/1.2.12
conan lock create --requires=zlib/1.2.13
conan lock create --requires=zlib/1.3
conan lock create --requires=zlib/1.3.1

produces this lock file:

{
    "version": "0.5",
    "requires": [
        "zlib/1.3.1#cac0f6daea041b0ccf42934163defb20%1765284699.337",
        "zlib/1.3#b3b71bfe8dd07abc7b82ff2bd0eac021%1733936247.661",
        "zlib/1.2.13#9df41c65e2c2b6ef47633dc32e0b699a%1743582310.97",
        "zlib/1.2.12#2ea72a0bae8b680f8a282e103e0a6880%1743582313.324",
        "zlib/1.2.11#bfceb3f8904b735f75c2b0df5713b1e6%1743582315.558"
    ],
    "build_requires": [],
    "python_requires": [],
    "config_requires": []
}

without any errors. Is this valid lock file?

[andrey-zherikov]

for pkg in $pkg_list; do conan lock create --requires=$pkg ...; done

This is doable without iteration with:

conan lock create --requires=zlib/1.3.1 --requires="bzip2/[*]" --requires=...

I will need to iterate to create a command line 😉