[question] Jenkins replay of successful build - cannot be found in lockfile error

[radonish]

I'm having an odd problem that I'm hoping can be chalked up to some silly user error.

At a high level, a parent job is:

1. Creating a lockfile
2. Uploading lockfile to Artifactory
3. Passing the path to the lockfile in Artifactory to the child job that will build the specific package

In a normal flow everything works. The child job uses the lockfile - it can both conan install and conan create successfully using the lockfile. The child job uploads the package to Artifactory successfully. The child job additionally runs unit tests.

We have some random unit test failures due to timing issues. For this reason, we sometimes go to the child job and do a Jenkins "replay" to force the build to occur again so that the unit tests will rerun. This time, when the child job is replayed with the same exact lockfile, the conan install results in a "ERROR: Build-require 'MyBuildRequire' cannot be found in lockfile". It's a confusing error as that build requirement is not in the lockfile for the initial build instance that had succeeded.

I can go and manually download the lockfile, attempt the build, and recreate the same error via command line. Further, if I skip the conan install step (done for the unit test stage needs) and go straight to conan create for the package, I get another cryptic error: "ERROR: Attempt to modify locked MyApp/1.0.0@myuser/mychannel#hash to MyApp/1.0.0@myuser/mychannel#hash". The package reference and hash is identical on both sides of the "to" in the error.

What is happening differently in the replay/rebuild flow that I'm describing here?

Thanks in advance!

• [x] I've read the CONTRIBUTING guide.

[memsharded]

Hi @radonish

I suspect that the lockfile that is being downloaded from Artifactory is mutated by the process itself. When a lockfile is incomplete, for example, because it locks a version, but it doesn't lock the package binary (package revision), because the binary hasn't been built yet, using that lockfile as input and output of some install/create command will fill that information in the lockfile. Subsequent install/create can fail because it doesn't make sense to build something that is already locked.

I would recommend trying:

• Making sure that a fresh lockfile is obtained at the beginning of the replay too
• If not, try to make sure that the input and output lockfiles of the install/create are different, until everything is green
• It might be useful to check the diff of the lockfile before and after the command, to understand what is changing in the lockfile when being used.

[radonish]

@memsharded, thanks for your response. In this case the lockfile generated by the parent job is fully specified; each package in the dependency graph has the package revision filled out (using package_revision_mode). The process on the parent job is currently:

1. Parent job does a conan export of each package contained in the repo
2. Parent job generates the lockfile with all package IDs and package revisions filled out
3. Parent job uploads fully specified lockfile to Artifactory

The process on the child job is:

1. Child job downloads lockfile from Artifactory
2. Child job executes conan create . user/channel --lockfile <lockfile downloaded from Artifactory> within the repo's package directory

The child job is not producing an output lockfile and is not uploading a change.

Since the lockfile is not mutated by the child and is fully specified by the parent, I would have expected the replay to just work.

[memsharded]

Sorry @radonish this was not followed up back then.

This referred to old Conan 1.X lockfiles. Lockfiles have been fully revamped in Conan 2.0, I am closing this ticket as outdated, please create new tickets referring to the new lockfiles for any further question or issue. Thanks!