[question] python_requires attributes and additional files

[aagor]

What is your question?

Hi,

I have two questions regarding python_requires packages. For context: We have a bunch of Conan packages that are build in the same way. They use the same CMakeLists.txt and the same conanfile.py. Everything that's different for these packages is handled by Conan/CMake internally (e.g. via set_name() and set_version() and similar stuff). In order to not duplicate that, a very minimal conanfile.py should be used for such stuff, the logic (for both Conan and CMake) should be imported from a central place (another Conan package) instead.

• The attributes seems to be inherited, even if docs say they aren't. Is that expected? Should I put it into the init() method nevertheless? From python_requires: Extending base classes:

In general, base class attributes are not inherited, and should be avoided as much as possible.

• However, this seems to work without problems in my case (see below).
• Can a python_requires package also contain some additional files? (like the mentioned CMakeLists.txt). Putting that in a separate package (with package_type = "build-scripts" would just complicate things, as the versions must match and that needs additional thoughts. The package_info() stuff should work. Maybe couple with build_policy = "missing"? (I know conan create doesn't build a package and there are no settings/options, but that's not required for my build scripts anyway.)

conanfile.py of base/1.0:

from conan import ConanFile

class GenericRecipe:
    user = "our-user"

    description = "our description"
    license = "our license"

    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False], "fPIC": [True, False]}
    default_options = {"shared": True, "fPIC": True}

    exports_sources = "<some stuff to export>"

    def export(self):
        # ...

    # other methods follow here ...

class Recipe(ConanFile):
    name = "base"
    version = "1.0"
    package_type = "python-require"

conanfile.py at every location:

from conan import ConanFile

class Recipe(ConanFile):
    python_requires = "base/1.0"
    python_requires_extend = "base.GenericRecipe"

Have you read the CONTRIBUTING guide?

• [x] I've read the CONTRIBUTING guide

[memsharded]

Hi @aagor Thanks for the question.

The attributes seems to be inherited, even if docs say they aren't. Is that expected? Should I put it into the init() method nevertheless?

You are right, and probably the docs could be improved. It looks it meant probably the "reference" attributes, like name, version, user and channel that could create problems if inherited, or other attributes that Conan does some processing over them, like settings initial list/tuple that is converted to a special object with extra logic after conanfile.py loading.

It is likely that simple attributes like license won't have any problems.

But in general it would be mostly a design recommendation, our experience is that inheritance of some attributes might sometimes be unexpected or have unexpected side issues.

I'll move this ticket to the docs for some clarification.

Can a python_requires package also contain some additional files? (like the mentioned CMakeLists.txt). Putting that in a separate package (with package_type = "build-scripts" would just complicate things, as the versions must match and that needs additional thoughts. The package_info() stuff should work. Maybe couple with build_policy = "missing"?

Let me ask another question. Lets say that you have a CMakeLists.txt that is common to all your projects. Did you ever consider other ways to reuse it, like for example using git-submodules? Why didn't you do it? Abstractions and reusage always have a cost. If we are talking about the root project CMakeLists.txt that for those projects it will no longer be there, because it will dynamically be retrieved from a Conan python-requires, are you sure that this will not impact negatively developer UX? Will you make the generate() method of the recipe to always copy the CMakeLists.txt from the python-requires to the local folder so the project can be built? And the local project will be doing a .gitignore for the CMakeLists.txt, because that copied file has to be ignored?

It might be possible to put other files inside the python_requires package and copy them from there into the consumers folder, it would probably be enough to make sure it is exported with export and then the consumer, you can see there is access through the .path attribute, something like:

          class MyConanfileBase(ConanFile):
                python_requires = "tool/0.1@user/channel"
                def source(self):
                    sources = self.python_requires["tool"].path
                    file_h = os.path.join(sources, "file.h")
                    # copy the file from the python-require to the current self.source_folder

While doable, my concerns here are more about the trade-offs and the cost of the abstraction.

[aagor]

Hi @memsharded, thanks for your answer.

It looks it meant probably the "reference" attributes, like name, version, user and channel that could create problems if inherited, or other attributes that Conan does some processing over them, like settings initial list/tuple that is converted to a special object with extra logic after conanfile.py loading.

I did both of these cases above, setting user and settings and it works. The docs say:

The resulting inheritance is equivalent to declare our Pkg class as class Pkg(pyreq.MyBase, ConanFile).

And this inheritance of attributes shouldn't be a problem at Python. Therefore I'm wondering what the restrictions really are, if any?

Will you make the generate() method of the recipe to always copy the CMakeLists.txt from the python-requires to the local folder so the project can be built?

Exactly.

[memsharded]

And this inheritance of attributes shouldn't be a problem at Python. Therefore I'm wondering what the restrictions really are, if any?

It seems you are correct, and the processing of attributes was kind of simplified in Conan 2 and it is now no longer having those issues to inherit attributes. I guess the documentation can be relaxed there a bit about this point.

Exactly.

Ok, I think it sounds doable from the Conan point of view, I don't see anything specially concerning there if you want to follow the above outlined approach.

[memsharded]

Any further question or comment here, or maybe this ticket can be closed now? Thanks for the feedback!

[aagor]

Hi, no, I'm fine with the current state. Thanks for your help.

You may close the ticket or move it to docs as you said above to make it easier for others, too.

[memsharded]

Thanks for the feedback, moving to the docs repo then