zesterer
`use chumsky;` causes linker error with latest Windows MSVC toolchain
This is not necessarily chumsky's fault, but I felt it was interesting to loop the community in on a Rust upstream issue that affects people particularly using chumsky: https://github.com/rust-lang/rust/issues/141626#issuecomment-2919419236
Inlining the above linked comment:
With 19.44 (VS 17.14), the linker is no longer happy about the project and crashes due to running out of memory when it tries to add a giant symbol to the PDB. I'm not exaggerating with "giant" - the use chumsky makes the project end up with two symbol names that are 73054 and 72828 bytes long.
Possible workarounds include
disabling debug information generation via
-Clink-arg=/DEBUG:NONE, either inRUSTFLAGSor a build script (you won't have PDBs).
-Csymbol-mangling-version=v0, either inRUSTFLAGSor a build script - this changes symbol name mangling so that it doesn't run into the bug.downgrading to 17.13.
That is unfortunate. For what it's worth, this can be worked around through use of the .boxed() combinator and also less overzealous use of .or in favour of choice. Thanks for the PSA though!
Hi @zesterer and everyone,
I'm adding another data point to this issue as I've just spent a significant amount of time debugging what appears to be the exact same problem. Thank you @ErichDonGubler for opening this!
My Experience: I'm working on a project using an operator precedence pyramid with chumsky, and I consistently hit the LNK1318: Unexpected PDB error on the latest Windows MSVC toolchain (stable-x86_64-pc-windows-msvc).
Reproducible Repository: I have created a repository that reliably reproduces this bug after a clean git clone: https://github.com/Karesis/cscript_ref/tree/bug/msvc-linker-LNK1318#
Key Findings:
The project uses edition = "2024", chumsky, logos, and miette.
The bug is 100% reproducible on MSVC, even after cargo clean and deleting Cargo.lock.
A friend of mine compiled the same repository on their machine (using Clang/LD), and they received a different but related error: ld: Assertion failed: (name.size() <= maxLength), file SymbolString.cpp. This strongly confirms the "giant symbol name" theory.
I have tried adding .boxed() at various points in my parser, including the end of the entire expression parser chain, but it was not sufficient to resolve the linking error in my project.
Workaround Confirmation: I can confirm that switching my project to the stable-x86_64-pc-windows-gnu toolchain completely resolves the issue, and the project compiles and runs successfully.
Hope this information and the reproducible repository are helpful for tracking the impact of the upstream Rust issue. Thanks for the amazing library!
@Karesis That's unfortunate. I would like to provide a good solution for this, but I can't think of any, without reverting to hacks (like shortening type names in an attempt to reduce the size of overall types).
In the example repository you provided, I notice that there is rather a long chain of parsers for the precedence climbing logic. Throwing a .boxed() at the end of each of these might go some way toward breaking up the type name size.
In general, this issue should be entirely possible to work around with enough .boxed()s, although I don't for one moment claim that this is an adequate solution given the possible performance overhead and inconvenience of doing so: but it is at least not game-breaking.
@Karesis That's unfortunate. I would like to provide a good solution for this, but I can't think of any, without reverting to hacks (like shortening type names in an attempt to reduce the size of overall types).
In the example repository you provided, I notice that there is rather a long chain of parsers for the precedence climbing logic. Throwing a
.boxed()at the end of each of these might go some way toward breaking up the type name size.In general, this issue should be entirely possible to work around with enough
.boxed()s, although I don't for one moment claim that this is an adequate solution given the possible performance overhead and inconvenience of doing so: but it is at least not game-breaking.
Thanks, that makes sense. I'll add more .boxed() calls to see if it fixes the issue.
Just to confirm, does this mean I need to find a balance between the performance cost of .boxed() and using just enough of them to resolve the linker error?
Just to confirm, does this mean I need to find a balance between the performance cost of .boxed() and using just enough of them to resolve the linker error?
Somewhat. In practice, LLVM might end up being able to devirtualise the required calls, resulting in zero meaningful performance overhead - but of course this is a much weaker guarantee than simple static dispatch.
My advice would be to set yourself a performance target and try to hit that. I imagine that in practice your parser will be more than fast enough anyway with a few .boxed()s arounds: the boxing is happening when the parser gets built, not during parsing, so the performance overhead is already pretty tiny anyway.
Just to confirm, does this mean I need to find a balance between the performance cost of .boxed() and using just enough of them to resolve the linker error?
Somewhat. In practice, LLVM might end up being able to devirtualise the required calls, resulting in zero meaningful performance overhead - but of course this is a much weaker guarantee than simple static dispatch.
My advice would be to set yourself a performance target and try to hit that. I imagine that in practice your parser will be more than fast enough anyway with a few
.boxed()s arounds: the boxing is happening when the parser gets built, not during parsing, so the performance overhead is already pretty tiny anyway.
yeah！got it
As a workaround I have resorted to using lld instead of MSVC's own linker by adding the following to .cargo/config.toml:
[target.x86_64-pc-windows-msvc]
linker = "rust-lld"
For the record, the workaround recommended by Microsoft seems to be the following:
[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "symbol-mangling-version=v0"]
Note that changing to the v0 mangling format may lead to demangling problems in some debuggers, see https://github.com/rust-lang/rust/issues/141626#issuecomment-3193832256.
