conan-io
[question] deploy vs deployer
What is your question?
I'm trying to understand deployment from the conan cache to final systems and I'm confused on the interaction of the deploy function in a conanfile and deployers. I have a conanfile with a custom deploy function that updates some of the files and I want to run a deployer such as runtime_deploy.
First off, why is calling the conanfile deploy function optional? I have to specify which packages I want via the deployer-package argument. I might be missing something but in my case, the function is required for the package to work post deployment. So it is an unpleasant surprise to the user that it is required.
Second, it looks like the deploy function and the runtime_deploy run independently. If I run conan install --requires mypkg/1.0.0 --deployer-package=mypkg --deployer=runtime_deploy. It looks like it calls my deploy function and then later calls the runtime_deployer which overwrites the files from my deploy function.
I am confused as to how these should (or shouldn't) interact. Is there a better way to accomplish what I am trying to do?
Have you read the CONTRIBUTING guide?
- [X] I've read the CONTRIBUTING guide
Hi @jsallay
Thanks for your questions.
First off, why is calling the conanfile deploy function optional? I have to specify which packages I want via the deployer-package argument. I might be missing something but in my case, the function is required for the package to work post deployment. So it is an unpleasant surprise to the user that it is required.
The deployers and deploy are designed to be always optional, it is something to extract things from the Conan cache, for final deploy outside of Conan. But the regular install, create flows should be able to work without deploying. For a package that requires some local customizations to work, but those customizations can happen in the Conan cache ,controlled by Conan, that is the finalize() method we are discussing in the other thread for Rpaths issues. But a package in general should be usable and consumable without deploying it at all.
The rule is more or less the opposite: the principle of "least surprise" of Conan is that any conan install or similar command will not do anything outside of the Conan cache (and generate local files as local generators define), but it shouldn't be putting files anywhere else in the system, something that deployers and deploy() can do. So for Conan, having such a behavior should always be explicit by the user. Something similar why the system_packages require a specific conf to be active to install things in the system, this should never happen without explicit user opt-in.
Second, it looks like the deploy function and the runtime_deploy run independently. If I run conan install --requires mypkg/1.0.0 --deployer-package=mypkg --deployer=runtime_deploy. It looks like it calls my deploy function and then later calls the runtime_deployer which overwrites the files from my deploy function.
The rule in this case is the one that "the most downstream consumer has precedence". In this case the deploy() methods of recipes are more upstream, they are way more difficult to change than changing a local mydeployer.py python script that is in the user side. So whatever --deployer=mydeployer should have precedence over the recipes deploy(), so it can modify, adapt, or change things done by deploy() methods. In this case the precedence means executing last, so it has full control of the final output.
Please let me know if this clarifies a bit the behavior. Thanks for the feedback!
Thanks for explaining it, I think that makes sense. I always call deploy with specific deploy arguments like --deployer-folder. So I hadn't considered the case of installing without wanting to deploy vs deploying to the local folder. There isn't a way to tell the user intention without a command line argument - hence the --deployer-package.
As far as the two pieces working together, I had assumed incorrectly that the runtime_deploy would call my conanfile's deploy method. I can see why this wouldn't work with how things are currently setup, but it might be nice feature.
So I hadn't considered the case of installing without wanting to deploy vs deploying to the local folder. There isn't a way to tell the user intention without a command line argument - hence the --deployer-package.
Yes, this is by far the most common use case, installing packages in the Conan cache and using them from there, without deploying. Like 99% of the usage is this way, and deployment is only done by some users at the very last mile when they need to deploy the final artifacts by other means (creating an installer for the customer, or a .deb package to deploy in servers).
As far as the two pieces working together, I had assumed incorrectly that the runtime_deploy would call my conanfile's deploy method. I can see why this wouldn't work with how things are currently setup, but it might be nice feature.
Yes, they are not really connected. I am not sure if connecting them would be that great, as there are many different corner cases, like for example what if calling several deployers like --deployer=runtime_deploy --deployer=license_deploy, they could end calling twice the recipes deploy() with potential collisions. These features aim to be quite orthogonal
I think my confusion comes from not understanding that these are orthogonal features. It is confusing since they both contain deploy. I think it would be helpful to update the documentation to reflect this: Something like:
The deploy function is an independent feature from a deployer. If a conan install command contains --deployer-package and --deployer arguments, the deploy functions of the selected package conanfiles will be run prior to the deployers.
Also, I understand the use case of installing into the cache without wanting to deploy anywhere. That is what I use the majority of the time as well. What I hadn't considered was deploying without using the --deployer-folder argument. I had assumed that a different argument would always be present to indicate that we were deploying to a different location.
Thanks for the feedback @jsallay, sorry that I didn't follow up earlier. It seems that the docs could get some improvements, I am moving this ticket to the docs repo then, but please let me know if you have any further question or issue. Thanks!