[question] conan 2 Why is layout influencing the copy in packge() when export-pkg?

[forry]

What is your question?

Hi, we had a simple package recipe:

import os
from conan.tools.files import copy
from conan import ConanFile

class QtNodesConan(ConanFile):
    name = "qtnodes"
    ...
    settings = "os", "compiler", "arch"
    package_type = "shared-library"
    build_policy = "never" # current conan doesn't allow for building locally in our setup

    def configure(self):
        self.settings.rm_safe("compiler.cppstd")

    def package(self):
        copy(self, "*", "./install/", self.package_folder)

    def package_info(self):
        self.cpp_info.libs = ["QtNodes"]

On CI we are using something like:

cmake --build ./build --target INSTALL --config Release
conan export-pkg . --user lexocad --channel prebuild --version ${env:FT_VERSION_NUMBER} -pr=lx

This worked fine until my colleague needed to do some local deployment where we usually use out-of-source builds and he added a layout so he could use editable mode:

    def layout(self):
        self.folders.build = "../build"       

        # In the local folder (when the package is in development, or "editable") the artifacts can be found:
        self.cpp.build.libdirs = ["lib/RelWithDebInfo"]
        self.cpp.build.libs = ["QtNodes"]

With this, the CI build stopped working because it seems the layout is interfering with the copy function in the package invoked by conan export-pkg. I have looked into the documentation for layout, copy tool, and export-pkg but I have not found any mention of this interaction.

Why is it interfering? Can it be changed somehow? What is the prefix for the copy tool's relative path, could I somehow simply do something there so it works with the layout function defined here?

Thanks

conan version 2.12.2

Have you read the CONTRIBUTING guide?

• [ ] I've read the CONTRIBUTING guide

[memsharded]

Hi @forry

Thanks for your question.

This worked fine until my colleague needed to do some local deployment where we usually use out-of-source builds and he added a layout so he could use editable mode:

This is not very clear, so the recipe then added to the generate() method, and maybe the build(). The conan export-pkg with build_policy = "never" is intended mostly for pre-compiled binaries, but not simultaneously used with editable mode.

The key issue is:

copy(self, "*", "./install/", self.package_folder)

Is using a relative path the src="./install" folder. This relative path will work on the current cwd directory, which for the package() method, is the self.build_folder directory.

If layout() adds a self.folders.build = "../build" it is effectively changing the self.build_folder to be in a different place.

You can pass an absolute path to the copy() function, such as copy(...., src=os.path.join(self.recipe_folder, <rel-path-here>) if that is what you want. In fact, in general, it is recommended to define the src and dst arguments of copy() as absolute paths, not relative.

[forry]

Thanks, we will try it. But it would be wonderful if the exact behaviour of the copy tool was documented. My instinct was that relative paths are relative to the working directory - where I run the conan from. But the truth is that I couldn't remember that being in the documentation either and it is a bit more complicated when the conan create takes the recipe and then runs it from to local cache. So for clarity to document, what each command exactly does and where in each phase, would be welcome. I remember that in conan 1 there was at least a list of methods that are run from the recipe for each command but to know from where (which working dir) they are run would also be nice.

[memsharded]

I remember that in conan 1 there was at least a list of methods that are run from the recipe for each command but to know from where (which working dir) they are run would also be nice.

Yes, that method order is also in Conan 2 in https://docs.conan.io/2/reference/commands/create.html#methods-execution-order

but to know from where (which working dir) they are run would also be nice.

The truth is that we have tried in the docs to use absolute paths, because it is way more unambiguous and explicit, no space for interpretation or having to think about it: https://docs.conan.io/2/reference/conanfile/methods/package.html, and as you commented, it is also very important to also abstract away the local-build vs cache-build different locations.

[forry]

Yes, that method order is also in Conan 2 in https://docs.conan.io/2/reference/commands/create.html#methods-execution-order

But for export-pkg it was not apparent that the layout method is also run. Also, the documentation seems to be unclear about what happens if the relative path is used. But, indeed, at the end of the package() page there is an indirect hint that the layout is run. But it is like looking for a needle in a haystack. It would help if that information was also fully available from export-pkg page. Also since, I had a problem with the copy I would expect that the doc page for copy would give me the answer or at least point me in the right direction. But it is very scant.

Otherwise thanks for a swift reply.

[memsharded]

Make sense, let me open and move to the docs repo to try to eventually improve this. Thanks for the feedback!