src build on windows not working

[gregorsoll]

What version of protobuf and what language are you using? Version: main/v3.6.0/v3.5.0 etc. (NOTE: please try updating to the latest version of protoc/runtime possible beforehand to attempt to resolve your problem) Language: C++/Java/Python/C#/Ruby/PHP/Objective-C/Javascript

What operating system (Linux, Windows, ...) and version? Windows

What runtime / compiler are you using (e.g., python version or gcc version) VC 14.16.27023

What did you do? Steps to reproduce the behavior:

1. cloned src and submodules
2. installed bazel, c++ buildtools
3. bazel build :protoc :protobuf

What did you expect to see build completed

What did you see instead? C:/users/xyz/_bazel_xyz/y7fcwcs2/external/com_google_protobuf/src/google/protobuf/compiler/csharp/BUILD.bazel:22:11: Compiling src/google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc [for tool] failed: (Exit 2): cl.exe failed: error executing command (from target @com_google_protobuf//src/google/protobuf/compiler/csharp:csharp) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 65 arguments skipped) cl : Command line warning D9002 : ignoring unknown option '-std=c++14' external/com_google_protobuf/src/google/protobuf/compiler/csharp/csharp_repeated_enum_field.cc(31): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/csharp/csharp_repeated_enum_field.h': No such file or directory ERROR: C:/users/xyz/_bazel_xyz/y7fcwcs2/external/com_google_protobuf/src/google/protobuf/compiler/BUILD.bazel:74:11: Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool] failed: (Exit 2): cl.exe failed: error executing command (from target @com_google_protobuf//src/google/protobuf/compiler:command_line_interface) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 68 arguments skipped) cl : Command line warning D9002 : ignoring unknown option '-std=c++14' external/com_google_protobuf/src/google/protobuf/compiler/command_line_interface.cc(35): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/command_line_interface.h': No such file or directory ERROR: C:/users/xyz/_bazel_xyz/y7fcwcs2/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/BUILD.bazel:53:11: Compiling src/google/protobuf/compiler/objectivec/primitive_field.cc [for tool] failed: (Exit 2): cl.exe failed: error executing command (from target @com_google_protobuf//src/google/protobuf/compiler/objectivec:objectivec) C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 67 arguments skipped) cl : Command line warning D9002 : ignoring unknown option '-std=c++14' external/com_google_protobuf/src/google/protobuf/compiler/objectivec/primitive_field.cc(31): fatal error C1083: Cannot open include file: 'google/protobuf/compiler/objectivec/primitive_field.h': No such file or directory INFO: Elapsed time: 4.749s, Critical Path: 0.59s INFO: 29 processes: 29 internal. FAILED: Build did NOT complete successfully

Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).

Anything else we should know about your project / environment

[deannagarcia]

It looks like the protobuf version you're using is pretty old and is no longer supported. Can you upgrade to the newest version (23.2) and see if you still see the issue?

[laramiel]

I ran into the same issue trying to upgrade to 23.x and here's my diagnosis:

MSVC cl.exe has a limitation (https://developercommunity.visualstudio.com/t/clexe-compiler-driver-cannot-handle-long-file-path/975889) on filename lengths of 260 characters (MAX_PATH). There appears to be no workaround using UNC paths.

Within Protobuf there are a number of source files organized like this:

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/objectivec/BUILD.bazel

with rules like (abbreviated):

cc_library(
    name = "objectivec",
    srcs = [ ... ],
    hdrs = [
        ...
        "text_format_decode_data.h",
    ],
    include_prefix = "google/protobuf/compiler/objectivec",
)

When bazel encounters rules like this it constructs include paths to provide the include_prefix mapping similar to this:

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

Counting the hash, using fastbuild for the config, but not counting <bazel_outout_base> & <root workspace>, this filename is already at 215 characters.

That leaves a maximum of 40 characters for the other parts.

It's pretty easy to run out of that. The default output base for my username is >30 characters.

One option to fix this is to reorganize the src/google/protobuf/compiler files to remove the nesting.

One workaround which I am using now is to set a bazel startup option to gain that 30 characters back: --output_base=C:\O, but that's probably not a great long-term solution.

See related bazel issues:

https://github.com/bazelbuild/bazel/issues/15961 https://github.com/bazelbuild/bazel/issues/4149 https://github.com/bazelbuild/bazel/issues/18683

[laramiel]

I tried to rename my dependency from com_google_protobuf to protobuf. There are two things which encumber that.

1. The built-in repository @bazel_tools references @com_google_protobuf://protoc. There does not appear to be a way to override the repo_mapping for this repository to make it reference something else. There is a repository https://github.com/bazelbuild/rules_proto which maybe bazel is using to externalize proto rules, but it doesn't seem very active. I'm not sure of the state of that.

2. grpc binds 'protobuf' already. This may not be the place to lament the style of the grpc bazel rules, but their extensive use of bind complicates my attempt to get a hermetic environment where my repository controls all the dependencies and does not allow dependencies to load third-party repositories.

So it turns out that it's a bit challenging to save characters simply by renaming the repository.

[hlopko]

@meteorcloudy could you recommend the best solution for us?

[meteorcloudy]

Using a shorter output base / output user root is the only direct workaround I can think of as recommended at https://bazel.build/configure/windows#long-path-issues

@oquenchil Is it possible to create the virtual_includes dir under some path that the length is O(1) instead of O(package path length)?

[hlopko]

CC @jskeet, would it be an option to not use include_prefix in the C# BUILD files at all?

[jskeet]

@hlopko: I'm afraid I'm really not the person to ask about this - my understanding of both C++ and Bazel is pretty low. I have no objection to it being changed, but it's not something I'd have confidence either doing or reviewing.

[meteorcloudy]

I guess we can at least try and see if it breaks any tests? ;)

[oquenchil]

@oquenchil Is it possible to create the virtual_includes dir under some path that the length is O(1) instead of O(package path length)?

I don't think so.

• Assuming source code cannot be changed the directory structure must be as listed on the include directive
• The symlink must be in a folder belonging to the package of the target creating the symlink

That doesn't leave out anything that can be shortened using a hash like we do for shared libraries on Windows.

I can't think of anything except what's already been mentioned: not using include_prefix or shorter output base.

[meteorcloudy]

The symlink must be in a folder belonging to the package of the target creating the symlink

This is currently also a package path right? Can we replace it with a hash?

Essentially

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

becomes

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/_virtual_includes/<hash of package path>/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

[laramiel]

I think that this would require a bazel change; basically bazel would need to be able to hand out such a directory with a shorter prefix. I don't know what that would entail.

I filed a feature request for that: https://github.com/bazelbuild/bazel/issues/18683

It was recategorized to c++ rules, which may not be a great category.

[hlopko]

Looking around, it seems that the reason to have all the 73 instances if include_prefix in the repo is easier compatibility with the Google internal clone of this repository. One solution here would be to fix header include paths in all C++ sources so we don't need include_prefix at all.

CC @mkruskal-google

[zhangyanjun2020]

I ran into the same issue trying to upgrade to 23.x and here's my diagnosis:

MSVC cl.exe has a limitation (https://developercommunity.visualstudio.com/t/clexe-compiler-driver-cannot-handle-long-file-path/975889) on filename lengths of 260 characters (MAX_PATH). There appears to be no workaround using UNC paths.

Within Protobuf there are a number of source files organized like this:

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/objectivec/BUILD.bazel

with rules like (abbreviated):

cc_library(
    name = "objectivec",
    srcs = [ ... ],
    hdrs = [
        ...
        "text_format_decode_data.h",
    ],
    include_prefix = "google/protobuf/compiler/objectivec",
)

When bazel encounters rules like this it constructs include paths to provide the include_prefix mapping similar to this:

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

Counting the hash, using fastbuild for the config, but not counting <bazel_outout_base> & <root workspace>, this filename is already at 215 characters.

That leaves a maximum of 40 characters for the other parts.

It's pretty easy to run out of that. The default output base for my username is >30 characters.

One option to fix this is to reorganize the src/google/protobuf/compiler files to remove the nesting.

One workaround which I am using now is to set a bazel startup option to gain that 30 characters back: --output_base=C:\O, but that's probably not a great long-term solution.

See related bazel issues:

bazelbuild/bazel#15961 bazelbuild/bazel#4149 bazelbuild/bazel#18683

I ran into the same issue trying to upgrade to 23.x and here's my diagnosis:

MSVC cl.exe has a limitation (https://developercommunity.visualstudio.com/t/clexe-compiler-driver-cannot-handle-long-file-path/975889) on filename lengths of 260 characters (MAX_PATH). There appears to be no workaround using UNC paths.

Within Protobuf there are a number of source files organized like this:

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/compiler/objectivec/BUILD.bazel

with rules like (abbreviated):

cc_library(
    name = "objectivec",
    srcs = [ ... ],
    hdrs = [
        ...
        "text_format_decode_data.h",
    ],
    include_prefix = "google/protobuf/compiler/objectivec",
)

When bazel encounters rules like this it constructs include paths to provide the include_prefix mapping similar to this:

<bazel_output_base>/<hash>/execroot/<root workspace>/bazel-out/x64_windows-<config>/bin/external/com_google_protobuf/src/google/protobuf/compiler/objectivec/_virtual_includes/objectivec/google/protobuf/compiler/objectivec/text_format_decode_data.h

Counting the hash, using fastbuild for the config, but not counting <bazel_outout_base> & <root workspace>, this filename is already at 215 characters.

That leaves a maximum of 40 characters for the other parts.

It's pretty easy to run out of that. The default output base for my username is >30 characters.

One option to fix this is to reorganize the src/google/protobuf/compiler files to remove the nesting.

One workaround which I am using now is to set a bazel startup option to gain that 30 characters back: --output_base=C:\O, but that's probably not a great long-term solution.

See related bazel issues:

bazelbuild/bazel#15961 bazelbuild/bazel#4149 bazelbuild/bazel#18683

i try it do work, thanks

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[freeform-andre]

I'm running into this issue on v28.3 even with a short output base. The only other cause I can think of is that I'm building without runfiles enabled as my windows environment does not have symlinks enabled. If there is a way to eliminate the virtual imports or fix whatever issue in upstream rules_cc that'd be fantastic.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[esrauchg]

The severe file length limits under by bazel+msvc are a known painpoint, not just for external users but also for the protobuf team's development itself (we have been warping our own file names and directory structure to keep our max length of paths to be approx 60 chars, with remainder of the 260 character limit is burned by other overhead).

Since we don't have any good way to mitigate this issue, and because bazel+msvc usage is low compared to other ways to build on Windows, we are planning to drop support for bazel+msvc in Protobuf 34 (the Q1 2026 release). Bazel 30 already disables support for this combination, with a flag to reenable support for it which gives time to gather feedback on this as well as give folks a longer horizon to migrate their builds to either make+msvc or bazel+clang-cl which don't suffer from this artificial limitation without being stuck on Protobuf 29 or lower:

https://protobuf.dev/news/v30/#poison-msvc--bazel

my windows environment does not have symlinks enabled

Unfortunately a lot of bazel behaviors are inherently symlink related (besides what was mentioned above, also e.g. strip_prefix on proto library), I don't think it'll be a winning battle to use bazel on windows without symlinks.

If you need to build on an env without symlinks, I think you probably want look into using Make builds on that environment; I'm actually unsure if it also will still have a different set of problems in that env, but at least the most obvious behaviors that are inherently symlink-related are in bazel behaviors.

[neilconway]

@esrauchg The engdoc.corp.google.com link you posted is not publicly accessible.

[esrauchg]

Apologies, accidentally used the incorrect url: corrected to https://protobuf.dev/news/v30/#poison-msvc--bazel

[crt-31]

and because bazel+msvc usage is low compared to other ways to build on Windows,

Um, I don't know if this is necessarily a proper statement. Regardless, bazel+msvc should be a supported configuration as it has been. Please don't 'drop support'... lets find a way forward, I know we can.

[esrauchg]

Please provide any feedback or questions about our ongoing bazel+msvc support on https://github.com/protocolbuffers/protobuf/issues/20085

Note that my short comment didn't reflect all of the effort and thinking that went into it before taking a decision to move in that direction: we do not take the decision lightly, but unfortunately have not been able to find any sufficient technical solution to the problem.

[github-actions[bot]]

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago. This issue will be closed and archived after 14 additional days without activity.

[zhangyanjun2020]

邮件已收到

[Mizux]

Should be fixed now see:

• https://github.com/protocolbuffers/protobuf/issues/20085

TLDR: Need Bazel=8.4.0+ and rules_cc=0.1.4+

[zhangyanjun2020]

邮件已收到