[question] Consuming/exporting libraries build using conan outside of conan

[kingudk]

trafficstars

Hi

We have started using conan to manage dependencies in a two C++ projects. One that creates a library and another one to consume it (along with other libraries; gtest for instance). The library is shared through our in-house artifactory repository. That is in good working order, but what I have not been able to piece together is how to consume or export the package to be consumed outside of conan. The specific usecase is that I need the library to be part of a java project (build using gradle), where we need two native libraries (one build for windows, another for linux). I would very much like to be able to fetch the pre-build libraries from artifactory as a dependency for the java project. So is there a way to consume the packages either directly from artifactory, or how can I "export" a package from my conan build? I've looked a bit at the 'deploy' generator but I can't get a clear picture if that is a viable way or how to use it. Any advise on the above?

[czoido]

Hi @kingudk,

Thanks a lot for you question. I think there are different ways to approach this problem. First, let me point you to this example in the Conan 2.0 documentation about integration with Android Studio and Conan that may inspire you in some way.

I think one way could be using the deploy generator to obtain a local copy of all your dependencies and include those in your Java project. As an idea you could invoke the conan install <whatever> -g deploy -r remote from the gradle script so that would take the libraries you need directly from your Artifactory so you can use them there.

Besides that, other idea could be creating a custom generator to generate a gradle script that adds the appropiate flags to the build system and point to the libraries in the Conan cache instead in the local folder.

Hope this helps.

[kingudk]

Hi @czoido

Thanks for responding :)

The conan install <whatever> -g deploy -r remote seems to be a partial viable path.

However it seem to insist on building the code on my platform which creates two issues:

1. requires the presence of compiler and cmake.
2. Won't let me get a Windows binary (dll) on a Linux host.

I.e. conan install <my package> -g deploy -r remote --build never -s:b os=Windows -s:b compiler='Visual Studio' -s:b compiler.version="17"

Gives me a .so-file, and not just the content of the conan_package.tgz. Is there something obvious that I'm missing? There is two packages: one build with GCC for Linux and one build with Visual Studio for Windows. And in the end I need both binaries.

[czoido]

Hi @kingudk,

If you upload the binaries to the repository using exactly the same profile than the one you are using to invoke the deploy generator if should work even if you are invoking the conan install from a different platform that the binaries you want to install.

Also, in the conan install there's no need to use the settings build, what you want is using settings from host that is the platform you want your binaries to run into.

Imagine that you want to deploy lib/1.0 and you did:

in Windows: conan create . -pr=./profile_windows && conan upload lib/1.0 -c --all in Linux: conan create . -pr=./profile_linux && conan upload lib/1.0 -c --all

Then in a third machine can be Linux, for example:

conan install lib/1.0@ -pr=./profile_windows -g deploy
conan install lib/1.0@ -pr=./profile_linux -g deploy

Should install what you want and not try to build anything.

I wanted to also comment that for 2.0 we are removing the deploy generator and substitute it by a new deployers mechanism to copy artifacts from the cache to user folders, and consume those copies while building. This is not documented yet, but I'll open an issue to document it as soon as possible ping you as soon as it is documented. It will be used like something similar to this in Conan 2.0:

conan install --require=hello/1.0@ --deploy=full_deploy -of=output

But you will be able to completely customize the deployment as a plugin.

Hope this helps.

[kingudk]

Hi @czoido

I did something along those lines. The two packages were build on two different hosts (docker images on windows and linux) and uploaded to our internal artifactory repo. On both hosts the "default" profile was used.

Conan can find both packages: `bash-4.4$ conan search mylib/1.0.0@user/chan -r myremote Existing packages for recipe mylib/1.0.0@user/chan:

Existing recipe in remote 'myremote':

Package_ID: 1311836d6737fc17f9eb791aac1075d44c057dea
    [options]
        shared: True
    [settings]
        arch: x86_64
        build_type: Release
        compiler: Visual Studio
        compiler.runtime: MD
        compiler.version: 17
        os: Windows
    [requires]
        gtest/1.11.0:10f3c0a3a24a1055f3670e6694da4a93d078e663
    Outdated from recipe: False

Package_ID: 51c3f50d6e470be56f158b48042b226121678064
    [options]
        fPIC: False
        shared: True
    [settings]
        arch: x86_64
        build_type: Release
        compiler: gcc
        compiler.libcxx: libstdc++
        compiler.version: 8
        os: Linux
    [requires]
        gtest/1.11.0:eea471029afd178285a9408339b804914029020f
    Outdated from recipe: True`

Extracting the default profile from a windows host and attempting to fetch the pre-build package results in the following: `bash-4.4$ conan install mylib/1.0.0@user/chan -g deploy -r myremote -pr=/src/csg-esm/winprofile Configuration: [settings] arch=x86_64 arch_build=x86_64 build_type=Release compiler=Visual Studio compiler.runtime=MD compiler.version=17 os=Windows os_build=Windows [options] [build_requires] [env]

mylib/1.0.0@user/chan: Retrieving from server 'myremote' mylib/1.0.0@user/chan: Trying with 'myremote'... Downloading conanmanifest.txt completed [1.11k]
Downloading conanfile.py completed [1.52k]
mylib/1.0.0@user/chan: Downloaded recipe revision 0 mylib/1.0.0@user/chan: Building package from source as defined by build_policy='missing' Installing package: mylib/1.0.0 Requirements mylib/1.0.0@user/chan from 'myremote' - Downloaded Packages mylib/1.0.0@user/chan:e9a552ebe8f994398de9ceee972f0ad207df0658 - Build

Installing (downloading, building) binaries... Downloading conan_sources.tgz completed [15.48k]
Decompressing conan_sources.tgz completed [0.00k]
mylib/1.0.0@user/chan: Configuring sources in /.conan/data/mylib/1.0.0/user/chan/source/. mylib/1.0.0@user/chan: Copying sources to build folder mylib/1.0.0@user/chan: Building your package in /.conan/data/mylib/1.0.0/user/chan/build/e9a552ebe8f994398de9ceee972f0ad207df0658 mylib/1.0.0@user/chan: Generator txt created conanbuildinfo.txt mylib/1.0.0@user/chan: Calling generate() mylib/1.0.0@user/chan: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior ERROR: mylib/1.0.0@user/chan: Error in generate() method, line 37 tc.generate() ConanException: VS non-existing installation: Visual Studio 17 `

Perhaps I should mention that we currently use conan v1.50.

Any idea of why it insists on building the package?

[czoido]

Hi @kingudk,

I think it could be because the shared option set to True in the remote package?

Maybe trying something like:

conan install mylib/1.0.0@user/chan -g deploy -r myremote -pr=/src/csg-esm/winprofile -o mylib:shared=True

[kingudk]

Hi @czoido

That made no difference, still trying to compile the package :( I tried both o, o:b and o:h and both with the package, package and version with user/chan. Still the same, it tries to compile the package. The conanfile.py that created the package in the first place have "build_policy = missing", so I wouldn't expect that to force a re-build.

[czoido]

Hi @kingudk,

Did you create those packages using the profile src/csg-esm/winprofile maybe something differs between the profile or settings and options you used to create and the ones you are using to install?

[kingudk]

Hi @czoido

Not that exact file. The package was deployed with using a docker image run on a Windows server 2022. The image started off by with an empty conan config (which just had our remote added by invoking conan remote add ... prior to running the build). The content of winprofile was taken from a Windows 11 VM with conan (same arch: x64). I'll see if I can make the image to generate a default profile and compare it with winprofile.

Update: The two default profiles are identical.