protobuf Correctly pick up & use `Protobuf_PROTOC_EXECUTABLE` in cross-compilation

It seems that when using find_package(Protobuf CONFIG), any eventual setting of Protobuf_PROTOC_EXECUTABLE is ignored.

This makes it hard for us in conda-forge, because find_package(Protobuf CONFIG) is necessary to avoid picking up CMake's own implementation of finding protobuf (and prefer an up-to-date protobuf-config.cmake; which is necessary for correctly linking abseil for example).

On the other hand, we simply must cross-compile on several architectures (e.g. no osx-arm64 runners at scale), and so if Protobuf_PROTOC_EXECUTABLE gets ignored (which carefully points to the build environment that matches the arch or the agent, and not the target environment for the final binary) we run into Bad CPU type in executable when it calls the wrong protoc.

The work-around we have come up with (and keep reinventing) in several places boils down to (courtesy @traversaro):

 find_package(Protobuf CONFIG)
 if(Protobuf_FOUND)
     message(STATUS "Found protobuf via cmake config")
+    if(Protobuf_PROTOC_EXECUTABLE)
+      set_target_properties(protobuf::protoc PROPERTIES
+        IMPORTED_LOCATION_RELEASE "${Protobuf_PROTOC_EXECUTABLE}"
+      )
+    endif()
 else()
     message(WARNING "Falling back to cmake FindProtobuf as Protobuf was not found via CONFIG")
     find_package(Protobuf REQUIRED)
 endif()

However, not everyone encountering this issue might be able to investigate and come up with this solution, so we keep running into it. I don't see why this shouldn't be supported out of the box, hence this issue.

What version of protobuf and what language are you using? Version: 4.24.3 Language: C++

What operating system (Linux, Windows, ...) and version?

linux/osx

What runtime / compiler are you using (e.g., python version or gcc version)

What did you do? Cross-compile a package that requires protoc, pass -DProtobuf_PROTOC_EXECUTABLE=... to CMake invocation

What did you expect to see

Correct protoc (corresponding to Protobuf_PROTOC_EXECUTABLE) gets used.

What did you see instead?

Wrong protoc gets picked up.

Anything else we should know about your project / environment

Some examples from conda-forge:

https://github.com/conda-forge/opencv-feedstock/commit/bbe46049a19e30c96cadbdf07affcaf0b8c6230d
https://github.com/conda-forge/protozfits-feedstock/commit/e752d336d2aee5a7626330d06f12084c42d94b2b

Nov 01 '23 11:11 h-vetinari

Thanks for reporting this! If you happen to know what needs to change to fix this I'd be happy to review a PR. I'm not very familiar with cross-compiling via cmake though, so this might take me some time to track down and fix

Nov 03 '23 23:11 mkruskal-google

@traversaro, is this something you think you could help with? 🙃

Nov 12 '23 11:11 h-vetinari

@traversaro, is this something you think you could help with? 🙃

Yes, for sure if there is interest upstream I can prepare a PR. As a preliminary analysis, after thinking a bit I am afraid of the the fine details on how we can do this "overloading" of the location and the unintended side-effects, especially when using multiple config CMake generators. Probably a simpler solution is to just make sure that Protobuf_PROTOC_EXECUTABLE by default is defined as protobuf::protoc, and that ${Protobuf_PROTOC_EXECUTABLE} is used in place of protobuf::protoc in https://github.com/protocolbuffers/protobuf/blob/92619cdd433c5eed314d6b871060ac2340f52906/cmake/protobuf-generate.cmake#L146 . The downside of this solution is that downstream projects that use protobuf::protoc instead of the protobuf_generate CMake function will not work out of the box, but anyhow the fix for those cases would be quite straightforward (use ${Protobuf_PROTOC_EXECUTABLE} in place of protobuf::protoc). To get an idea, we can do something similar to what we did in https://github.com/gazebosim/gz-msgs/pull/392 .

Nov 12 '23 21:11 traversaro

Setting Protobuf_PROTOC_EXECUTABLE used to work. Now it looks like the new variable is protobuf_PROTOC_EXE but as mentioned earlier it gets ignored

Nov 29 '23 16:11 ekigwana

It seems that when using find_package(Protobuf CONFIG), any eventual setting of Protobuf_PROTOC_EXECUTABLE is ignored.

This makes it hard for us in conda-forge, because find_package(Protobuf CONFIG) is necessary to avoid picking up CMake's own implementation of finding protobuf (and prefer an up-to-date protobuf-config.cmake; which is necessary for correctly linking abseil for example).

On the other hand, we simply must cross-compile on several architectures (e.g. no osx-arm64 runners at scale), and so if Protobuf_PROTOC_EXECUTABLE gets ignored (which carefully points to the build environment that matches the arch or the agent, and not the target environment for the final binary) we run into Bad CPU type in executable when it calls the wrong protoc.

The work-around we have come up with (and keep reinventing) in several places boils down to (courtesy @traversaro):
 find_package(Protobuf CONFIG)
 if(Protobuf_FOUND)
     message(STATUS "Found protobuf via cmake config")
+    if(Protobuf_PROTOC_EXECUTABLE)
+      set_target_properties(protobuf::protoc PROPERTIES
+        IMPORTED_LOCATION_RELEASE "${Protobuf_PROTOC_EXECUTABLE}"
+      )
+    endif()
 else()
     message(WARNING "Falling back to cmake FindProtobuf as Protobuf was not found via CONFIG")
     find_package(Protobuf REQUIRED)
 endif()
However, not everyone encountering this issue might be able to investigate and come up with this solution, so we keep running into it. I don't see why this shouldn't be supported out of the box, hence this issue.

What version of protobuf and what language are you using? Version: 4.24.3 Language: C++

What operating system (Linux, Windows, ...) and version?

linux/osx

What runtime / compiler are you using (e.g., python version or gcc version)

What did you do? Cross-compile a package that requires protoc, pass -DProtobuf_PROTOC_EXECUTABLE=... to CMake invocation

What did you expect to see

Correct protoc (corresponding to Protobuf_PROTOC_EXECUTABLE) gets used.

What did you see instead?

Wrong protoc gets picked up.

Anything else we should know about your project / environment

Some examples from conda-forge:

conda-forge/opencv-feedstock@bbe4604

conda-forge/protozfits-feedstock@e752d33

I had to set IMPORTED_LOCATION_RELWITHDEBINFO so even that seems like fun trap that just cost me 4 hours to discover

Nov 29 '23 18:11 ekigwana

I had to set IMPORTED_LOCATION_RELWITHDEBINFO so even that seems like fun trap that just cost me 4 hours to discover

Yes, I was referring exactly to that with "I am afraid of the the fine details on how we can do this "overloading" of the location and the unintended side-effects, especially when using multiple config CMake generators.", sorry about that. I hope to prepare a concrete proposal to address this in the next week.

Nov 30 '23 09:11 traversaro

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

Feb 28 '24 10:02 github-actions[bot]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

Ops, sorry about that. I forgot to follow up on my original " I hope to prepare a concrete proposal to address this in the next week.".

Feb 28 '24 10:02 traversaro

Any update over here? The suggested workaround doesnt work for me and it throws error saying i cant use set_target_properties on alias. I literally did the same thing as what has been told here!

Mar 18 '24 14:03 0xDivyanshu-new

Any update over here? The suggested workaround doesnt work for me and it throws error saying i cant use set_target_properties on alias. I literally did the same thing as what has been told here!

How are you including protobuf in your CMake project, via find_package(Protobuf) or add_subdirectory/FetchContent?

Mar 18 '24 14:03 traversaro

if(PROTOBUF_COMPILE)
    include(FetchContent)
    set(FETCHCONTENT_QUIET ON)
    set(FETCHCONTENT_UPDATES_DISCONNECTED ON)
    set(BUILD_SHARED_LIBS OFF)
    set(CMAKE_POSITION_INDEPENDENT_CODE ON)
    find_package(Git REQUIRED)
    set(protobuf_BUILD_TESTS OFF CACHE BOOL "" FORCE)
    set(protobuf_BUILD_CONFORMANCE OFF CACHE BOOL "" FORCE)
    set(protobuf_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
    set(protobuf_BUILD_EXPORT OFF CACHE BOOL "" FORCE)
    set(protobuf_WITH_ZLIB OFF CACHE BOOL "" FORCE)
    fetchcontent_declare(
        protobuf GIT_REPOSITORY "https://github.com/protocolbuffers/protobuf.git" GIT_TAG v3.15.6
        PATCH_COMMAND "" SOURCE_SUBDIR cmake
    )
    fetchcontent_makeavailable(protobuf)
    fetchcontent_getproperties(protobuf SOURCE_DIR PROTOBUF_INCLUDE_DIR)

else()
    set(PROTOBUF_INCLUDE_DIR "${CMAKE_CURRENT_LIST_DIR}/build/x64/_deps/protobuf-src")
endif(PROTOBUF_COMPILE)

set(PROTOBUF_INCLUDE_DIR "${PROTOBUF_INCLUDE_DIR}/src")
include_directories(${PROTOBUF_INCLUDE_DIR})
message("PROTOBUF_INCLUDE_DIR value : ${PROTOBUF_INCLUDE_DIR}")
set(Protobuf_INCLUDE_DIR ${PROTOBUF_INCLUDE_DIR})
set(Protobuf_LIBRARIES "${PROTOBUF_INCLUDE_DIR}/protobuf-build/libprotoc.lib")
set(Protobuf_PROTOC_EXECUTABLE "${PROTOBUF_INCLUDE_DIR}/protobuf-build/protoc.exe")

find_package(Protobuf REQUIRED)
set_target_properties(
    "protobuf::protoc" PROPERTIES IMPORTED_LOCATION_RELWITHDEBINFO ${Protobuf_LIBRARIES}
                                  IMPORTED_LOCATION_RELEASE ${Protobuf_PROTOC_EXECUTABLE}
)

I am doing fetchcontent in cmake and then using find_package

Mar 18 '24 15:03 0xDivyanshu-new

You need to use either FetchContent or use find_package, in general you can't mix the two.

Mar 18 '24 15:03 traversaro

But if I am to cross compile protobuf, i would have to build it in machine where cross compilation tools are available. In such cases, if i dont use FetchContent i wont be able to build it using my setup. And if i dont use find_package i wont be able to use protobuf_generate_cpp to generate the headers.

Mar 18 '24 15:03 0xDivyanshu-new

In the kind of cross-compilation builds to which the workaround in the OP refers, protobuf is cross-compiled and installed before you configure for cross-compilation the package that depends on protobuf, and this cross-compiled protobuf is found via find_package.

Mar 18 '24 15:03 traversaro

Since I'm keeping an eye on this issue, I'll also leave a revised version of OP's for the internet wanderers like me.

find_package(protobuf CONFIG REQUIRED)
if(CMAKE_CROSSCOMPILING)
    find_program(_PROTOBUF_PROTOC_EXECUTABLE protoc)
    if(NOT TARGET protobuf::protoc)
        add_executable(protobuf::protoc IMPORTED)
    endif()
    set_target_properties(protobuf::protoc PROPERTIES
        IMPORTED_LOCATION_RELEASE "${_PROTOBUF_PROTOC_EXECUTABLE}"
        IMPORTED_LOCATION_DEBUG "${_PROTOBUF_PROTOC_EXECUTABLE}"
        IMPORTED_LOCATION_RELWIHDEBINFO "${_PROTOBUF_PROTOC_EXECUTABLE}"
        IMPORTED_LOCATION_NOCONFIG "${_PROTOBUF_PROTOC_EXECUTABLE}"
    )
endif()

I'm checking if the protobuf::protoc target exists since cross-compiling protobuf with -Dprotobuf_BUILD_PROTOC_BINARIES=OFF (like I do) does not create the target
Added a bunch of IMPORTED_LOCATION_<CONFIG> since IMPORTED_LOCATION gets overriden by those

Apr 16 '24 15:04 dubvulture

Since I'm keeping an eye on this issue, I'll also leave a revised version of OP's for the internet wanderers like me.
find_package(protobuf CONFIG REQUIRED)
if(CMAKE_CROSSCOMPILING)
    find_program(_PROTOBUF_PROTOC_EXECUTABLE protoc)
    if(NOT TARGET protobuf::protoc)
        add_executable(protobuf::protoc IMPORTED)
    endif()
    set_target_properties(protobuf::protoc PROPERTIES
        IMPORTED_LOCATION_RELEASE "${_PROTOBUF_PROTOC_EXECUTABLE}"
        IMPORTED_LOCATION_DEBUG "${_PROTOBUF_PROTOC_EXECUTABLE}"
        IMPORTED_LOCATION_RELWIHDEBINFO "${_PROTOBUF_PROTOC_EXECUTABLE}"
        IMPORTED_LOCATION_NOCONFIG "${_PROTOBUF_PROTOC_EXECUTABLE}"
    )
endif()
I'm checking if the protobuf::protoc target exists since cross-compiling protobuf with -Dprotobuf_BUILD_PROTOC_BINARIES=OFF (like I do) does not create the target

Added a bunch of IMPORTED_LOCATION_<CONFIG> since IMPORTED_LOCATION gets overriden by those

Thank you, this works for me in yocto cross compilation. Any chances that this could be merged?

Jun 10 '24 08:06 rsniezek

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

Sep 08 '24 10:09 github-actions[bot]

The issue is still present.

Sep 08 '24 10:09 traversaro

I've combined and modified some of the workarounds posted above to optionally use protoc provided by vcpkg:

if(CMAKE_CROSSCOMPILING)
  if(NOT DEFINED ${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE)
    if(DEFINED VCPKG_HOST_TRIPLET)
      set(${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE
          "${VCPKG_INSTALLED_DIR}/${VCPKG_HOST_TRIPLET}/tools/protobuf/protoc"
      )
    else()
      find_program(
        ${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE
        protoc
        REQUIRED
      )
    endif()
  endif()

  if(NOT TARGET protobuf::protoc)
    add_executable(protobuf::protoc IMPORTED)
  endif()

  set_target_properties(
    protobuf::protoc
    PROPERTIES
      IMPORTED_LOCATION_RELEASE "${${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE}"
      IMPORTED_LOCATION_DEBUG "${${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE}"
      IMPORTED_LOCATION_RELWIHDEBINFO "${${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE}"
      IMPORTED_LOCATION_NOCONFIG "${${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE}"
  )

  message(STATUS "using protoc found at: ${${PROJECT_NAME}_PROTOBUF_PROTOC_EXECUTABLE}")
endif()

Sep 26 '24 13:09 justusranvier

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

Dec 26 '24 10:12 github-actions[bot]

Not stale

Dec 26 '24 12:12 h-vetinari

This is related, even if it does not solve the issue described here: https://github.com/protocolbuffers/protobuf/pull/17888/files .

Mar 01 '25 08:03 traversaro

CMake 4.0 was released with an ad-host solution for this exact same problem just for Python, in the form of the Python_ARTIFACTS_PREFIX , see https://gitlab.kitware.com/cmake/cmake/-/merge_requests/10157 and https://youtu.be/TESpxsilA1k?si=Jm5ACBd4-8dg5RPp&t=683 .

Interestingly, applying something similar to CMake config searches was proposed in https://gitlab.kitware.com/cmake/cmake/-/issues/18169, and the alternative solution suggested (that is similar to what Qt6 does, as far as I know) was:

The B project could be divided up to provide a host tools variant that is named differently.

So a possible solution would be to split protoc metadata in its own CMake config file, but that is non trivial to do, also considering the fact that both FindProtobuf.cmake (mantained in CMake) and protobuf-config.cmake (installed and mantained here) continue to exist and be mantained, see https://gitlab.kitware.com/cmake/cmake/-/issues/24321 .

Jun 05 '25 10:06 traversaro

Cross is also broken in the find module by this: https://gitlab.kitware.com/cmake/cmake/-/issues/23659

On Ubuntu 24, if you apt-install libprotobuf-dev , you might hit that too.

Aug 12 '25 02:08 Ryanf55

Does -DWITH_PROTOC=<protoc location> work? It looks like I added this 2 years ago specifically for cross-compilation in our CI, not sure why I didn't think of this back then

Aug 13 '25 02:08 mkruskal-google

Does -DWITH_PROTOC=<protoc location> work? It looks like I added this 2 years ago specifically for cross-compilation in our CI, not sure why I didn't think of this back then

I may be wrong, but at a first glance WITH_PROTOC is cmake variable used by the protobuf build system, not something exposed by protobuf-config.cmake and used in protobuf_generate CMake function by downstream projects that call find_package(Protobuf CONFIG REQUIRED).

Aug 18 '25 08:08 traversaro

WITH_PROTOC

It also doesn't exist in FindProtobuf.cmake. https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/FindProtobuf.cmake

On Ubuntu 24, there are no CMake config files distributed, just the pkgconfig file.

Aug 18 '25 13:08 Ryanf55

Ah sorry, if you're trying to use this with protobuf_generate doesn't https://github.com/protocolbuffers/protobuf/pull/17888 solve this then? You can pass a PROTOC_EXE argument overriding the protoc binary to use for codegen

Aug 18 '25 23:08 mkruskal-google

Thanks for the reference! That is a bit different from what was discussed in this issue, as it requires changes to the CMake source code of the project, while setting Protobuf_PROTOC_EXECUTABLE worked without any CMake modification in the downstream project. However, setting protobuf_generate_PROTOC_EXE from the CMake command line may work fine indeed, even if that variable is an internal variable of protobuf_generate so it is technically a bit dirty. @mkruskal-google Would you be open to have a properly defined and documented CMake variable to override the PROTOC_EXE argument of protobuf_generate from the CMake command line? I think that would permit to definitely close the issue.

@Ryanf55 I think the original intent of @h-vetinari was to track the problem on the find_package(Protobuf CONFIG) side on this issue, but the PROTOC_EXE was also added in upstream CMake in https://https://gitlab.kitware.com/cmake/cmake/-/merge_requests/9888 .

Aug 19 '25 13:08 traversaro

Thanks for the reference! That is a bit different from what was discussed in this issue, as it requires changes to the CMake source code of the project, while setting Protobuf_PROTOC_EXECUTABLE worked without nay CMake modification in the downstream project. However, setting protobuf_generate_PROTOC_EXE from the CMake command line may work fine indeed, even if that variable is an internal variable of protobuf_generate so it is technically a bit dirty. Would you be open to have a properly defined and documented CMake variable to override the PROTOC_EXE argument of protobuf_generate from the CMake command line? I think that would permit to definitely close the issue.

I tested using protobuf_generate_PROTOC_EXE in place of Protobuf_PROTOC_EXECUTABLE and the patches mentioned in the OP in https://github.com/conda-forge/protozfits-feedstock/pull/54, and indeed it works fine, except the fact that it is exploiting a undocumented implementation detail.

Aug 19 '25 14:08 traversaro