mold
mold copied to clipboard
rui314
Support LTO linker plugin
Similarly to #93, please consider adding -flto support for GCC bytecode.
Your linker looks very promising and LTO is getting more commonly used. Note there are distros like openSUSE, Fedora, (Ubuntu, Debian in near future), that use LTO by default for package builds.
Thanks.
Yes, I want to do that. My plan is to support the standard linker plugin API, so that mold works both LLVM and GCC. I just don't have enough time to do that at the moment.
I think it is difficult to have a plugin to support both LLVM and GCC. plugin-api.h style ld_plugin_symbol is very different from LLVM llvm::lto::SymbolResolution.
Both GCC and LLVM support the linker API (you need to build LLVM with buinultils include path to get the gold plugin). What would be really nice (and technically possible I think) would be to support multiple plugins at once. So if you have some object files bullt with GCC and other with LLVM you get LTO at least within the partitions of program built with the same compiler.
I tried to write code to support the LTO linker plugin only to find that the API is not suitable for mold. The linker plugin API is designed with the conventional linker design in mind, and mold is quite different from that.
Here is an example: the plugin API provide a function to ask if the plugin wants to "claim" the ownership of an object file. That function claims the ownership of an object file if the file was compiled with -flto. If the plugin takes an ownership of a file, it reads a symbol table of the file and calls a callback to notify to the linker the contents of the symbol table. Finally, the plugin compiles all files it owns and pass the result back to the linker.
So, in this API, there's no way to read a symbol table without including the file into the final LTO result. This API design doesn't work for us because mold reads object files from archive files even if they won't be included into the final result.
It looks like this blocks us from using the plugin API. We probably need to propose a new API to both GCC and LLVM. That's I guess not easy. My hope is that there might be a chance to get a favorable treatment by GCC as GNU doesn't have a faster linker anymore (gold is quasi-deprecated, and lld is unlikely to support the linker plugin API because it instead directly uses LLVM), so they might have an incentive on their side to make their compiler work well with mold.
Rui, thank you for looking into this. The linker plugin API is extensible (it has versions and it was extended in the past), So if needed, we can discuss a new version of the plugin API. I maintain the GCC LTO implementation and would be happy to help with this. Gold maintainers are Iant Taylor and Cary Countant. For BFD ld implementation I think H.J. Lu is good contact.
The reason mold reads all files in archive right away is to achieve parallelism of linking process and later it discards symbols that are not needed?
One need to keep in mind that LTO linking is relatively slow in the end (because all compilation needs to be done) so I am not sure how much benefits the parallelism gives. But I wonder if one can simply use multiple instances of the plugin - in the "pre-linking" stage one can invoke the plugin and check if a file is accepted by it and to obtain symbol tables. Similar usage of linker is done by nm, ar and other tools. Once the final set of files is determined feed the files again into a new instance of the plugin plugin and do the final job.
lto-plugin in GCC is quite cheap wrapper that does
- determine if file is GCC LTO file
- reads the symbol table and passes it to the linker
- once pre-linking is done it stores the symbol table given by linker to a resolution file
- it invokes lto-wrapper which eventually invokes GCC in a separate process. There is relatively little bookkeeping between 2) and 3) so I do not see it as much of a complication if some files got discarded between these two steps. Adding an extra LDPT method for discarding previously claimed files is IMO quite easy (if agreed on with the binutils ld&gold maintainers) and it would save some effort in duplicated loading of symbol tables. Perhaps it would also make it possible to make linker plugin thread safe to some extend to support faster prelinking.
Honza
On Tue, Jan 4, 2022 at 11:58 AM Rui Ueyama @.***> wrote:
I wrote code to support the LTO linker plugin only to find that the API is not suitable for mold. The linker plugin API is designed with the conventional linker design in mind, and mold is quite different from that.
Here is an example: the plugin API provide a function to ask if the plugin wants to "claim" the ownership of an object file. That function claims the ownership of an object file if the file was compiled with -flto. If the plugin takes an ownership of a file, it reads a symbol table of the file and calls a callback to notify to the linker the contents of the symbol table. Finally, the plugin compiles all files it owns and pass the result back to the linker.
So, in this API, there's no way to read a symbol table without including the file into the final LTO result. This API design doesn't work for us because mold reads object files from archive files even if they won't be included into the final result.
It looks like this blocks us from using the plugin API. We probably need to propose a new API to both GCC and LLVM. That's I guess not easy. My hope is that there might be a chance to get a favorable treatment by GCC as GNU doesn't have a faster linker anymore (gold is quasi-deprecated, and lld is unlikely to support the linker plugin API and instead directly uses LLVM), so they might have an incentive on their side to make their compiler to work well with mold.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1004710952, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW2DMLUGWXXFN6MTVPTUULHEVANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
Hi Honza,
Thank you for your comment! It was nice to have you here to discuss how to support GCC LTO support in mold.
We do read archive members right away for parallelism. We parse all input files (including the ones in archive files) simultaneously at the beginning of the process, so that the symbol resolution stage doesn't have to read any files from the disk. I understand this speedup doesn't matter much when LTO is in use, but this is how mold resolves symbols, and I don't think we want to implement a different algorithm only for LTO (not only to save the amount of code but also to guarantee that symbols are resolved in the exact same way both in a non-LTO and LTO builds).
So, yes, what I immediately needed is an API to discard previously claimed files.
But can we add a convenient API to read symbols from an LTO object file without claiming it instead? That would be more convenient than letting the plugin claim a file and later letting it forget about it.
On the GCC side I think adding an extra API hook reading symbol tables and not registering it should be easy. See claim_file_handler in lto-plugin.c. It essentially checks the file, then it calls add_symbols or add_symbols_v2 and then it adds the file to the list of claimed files. You essentially want to split this into two API calls - check_file_and_read_symbol_table and reigster_file it seems.
On Tue, Jan 4, 2022 at 2:25 PM Rui Ueyama @.***> wrote:
Hi Honza,
Thank you for your comment! It was nice to have you here to discuss how to support GCC LTO support in mold.
We do read archive members right away for parallelism. We parse all input files (including the ones in archive files) simultaneously at the beginning of the process, so that the symbol resolution stage doesn't have to read any files from the disk. I understand this speedup doesn't matter much when LTO is in use, but this is how mold resolves symbols, and I don't think we want to implement a different algorithm only for LTO (not only to save the amount of code but also to guarantee that symbols are resolved in the exact same way both in a non-LTO and LTO builds).
So, yes, what I immediately needed is an API to discard previously claimed files.
But can we add a convenient API to read symbols from an LTO object file without claiming it instead? That would be more convenient than let the plugin claim a file and later let it forget about it.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1004808872, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW7EIAWONBFYTVREUMDUULYLDANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
But can we add a convenient API to read symbols from an LTO object file without claiming it instead? That would be more convenient than letting the plugin claim a file and later letting it forget about it.
Hey. I've just implemented the hook ld_plugin_open_and_read_symbols_handler which returns true if an object contains LTO symbols. It's likely not going to work with LTO offloading, but we can take a look at the later:
https://gcc.gnu.org/wiki/Offloading
So please let's use the following GCC branch of testing and development: https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/devel/mold-lto-plugin
The branch will be pulled by default if you clone the repo.
@rui314 Can you please test it and provide feedback?
Martin, thanks for implementing this! It may be enough to get plugin API working for mold. For effectivity we will however eventually want ld_plugin_register_file to bypass the symtab reading which is done by claim_file_handler and only adds the filename to the list of objects to pass to lto-wrapper.
On Tue, Jan 4, 2022 at 4:42 PM Martin Liška @.***> wrote:
But can we add a convenient API to read symbols from an LTO object file without claiming it instead? That would be more convenient than letting the plugin claim a file and later letting it forget about it.
Hey. I've just implemented the hook ld_plugin_open_and_read_symbols_handler which returns true if an object contains LTO objects. It's likely not going to work with LTO offloading, but we can take a look at the later: https://gcc.gnu.org/wiki/Offloading
So please let's use the following GCC branch of testing and development:
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/devel/mold-lto-plugin
The branch will be pulled by default if you clone the repo.
@rui314 https://github.com/rui314 Can you please test it and provide feedback?
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1004917590, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW6NSOMA2GYLCW75XDLUUMINNANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
@marxin Let me experiment with your change. I'll get back to you in a few days.
For effectivity we will however eventually want ld_plugin_register_file to bypass the symtab reading which is done by claim_file_handler and only adds the filename to the list of objects to pass to lto-wrapper.
Sure, I'm planning to do that. It will be simple.
Looking forward to when mold will support LTO.
I was thinking maybe use gold as the LTO fallback for gcc but maybe bfd is better for compatibility.
I just realized it's ld.bfd not bfd.
It's there in Ubuntu and Termux just that the symlink is missing on Termux.
@joshcangit GNU ld is still the de-facto standard linker on Linux, so it is very unlikely that you don't have it but gold and lld on your system. What distro are you using? What ld --version says on your system?
@rui314 Hey. Have you find a time experimenting with GCC LTO plug-in support?
@marxin Not yet. I'm receiving more bug reports than I expected and I haven't had a chance to work on LTO yet.
@marxin I investigated more about the LTO linker plugin interface and found that our usage is actually supported by the standard API. The point is that even though we have to let the linker plugin to claim the ownership of an LTO object file to read its symbol table, we can later let it ignore any file by returning LDPT_NO_SYMS as a response from the get_symbols callback.
So it looks like we can do LTO with the existing API, which is a good news because we can support old versions of GCC and LLVM.
It's still work-in-progress, but with the above change, mold can now link itself with -flto. It works both with GCC and Clang.
These are all great news. I'm going to be off next week, so I'll start with intensive testing the week after that.
It looks like I can now build clang with LTO using clang and mold. But I can't do the same build using gcc and mold. It's because the GCC linker plugin is still using get_symbols_v2 API instead of get_symbols_v3 API, so there's no way to tell to the plugin that an archive member wasn't chosen to be included in the final linker output.
Looking at the plug-in, do I understand it correctly that what we need is to add get_symbols_v3 to GCC's LTO plug-in and skip resolution info emission if LDPT_NO_SYMS is returned by the linker. Am I correct?
Correct. I don't know if it will fix all the remaining problems, but that v3 API is clearly missing in the GCC LTO plugin. It is odd that GCC defined only the enums for the v3 API and didn't implement the feature.
It seems no one told us that the API was extended (and why:).
Looking at history it was added in 2016 by Evgenii Stepanov to support --start-lib and --end-lib and there is Raphael's email explaining bit of rationale https://binutils.sourceware.narkive.com/AhbbomII/linker-plugin-api-limitation It is not hard to support at the plugin side though life would be easier if we were informed about it :)
Until gcc picks the v3 extension (which may or may not be possible for gcc12.1 since we are in feature freeze), I wonder if it would work for you to also accept v1/v2 plugins to and simply return all symbols as preempted? It will make GCC to load the object file and parse their symbols tables, types and declarations for no good reason (which is wasteful and slow) but I do not see any immediate reason why that should be harmful to the final output.
Thanks for all the work on the plugin support. It is pretty impressive:) Honza
On Sun, Feb 13, 2022 at 7:19 PM Rui Ueyama @.***> wrote:
Correct. I don't know if it will fix all the remaining problems, but that v3 API is clearly missing in the GCC LTO plugin. It is odd that GCC defined only the enums for the v3 API and didn't implement the feature.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1038316037, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW2HPJ3OO5YNYYTR223U27Y2HANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
Martin, also in addition to skipping resolution file emission, we probably want to take the file away from lto1 command line, so it is not parsed (at least from my quick look at the code today)
On Sun, Feb 13, 2022 at 7:57 PM Jan Hubička @.***> wrote:
It seems no one told us that the API was extended (and why:).
Looking at history it was added in 2016 by Evgenii Stepanov to support --start-lib and --end-lib and there is Raphael's email explaining bit of rationale
https://binutils.sourceware.narkive.com/AhbbomII/linker-plugin-api-limitation It is not hard to support at the plugin side though life would be easier if we were informed about it :)
Until gcc picks the v3 extension (which may or may not be possible for gcc12.1 since we are in feature freeze), I wonder if it would work for you to also accept v1/v2 plugins to and simply return all symbols as preempted? It will make GCC to load the object file and parse their symbols tables, types and declarations for no good reason (which is wasteful and slow) but I do not see any immediate reason why that should be harmful to the final output.
Thanks for all the work on the plugin support. It is pretty impressive:) Honza
On Sun, Feb 13, 2022 at 7:19 PM Rui Ueyama @.***> wrote:
Correct. I don't know if it will fix all the remaining problems, but that v3 API is clearly missing in the GCC LTO plugin. It is odd that GCC defined only the enums for the v3 API and didn't implement the feature.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1038316037, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW2HPJ3OO5YNYYTR223U27Y2HANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
I want to support the v2 API, but it looks like setting LDPR_PREEMPTED_REG or LDPR_PREEMPTED_IR to all symbols is still different from returning LDPT_NO_SYMS. I don't know why, but looks like sometimes GCC LTO leaves such symbols as undefined symbols instead of completely ignoring them. Do you have any idea for workaround?
I suspect that if an IR object file contains a static initializer which refers some symbol, we can't eliminate a reference to that symbol even if we mark it as LDPR_PREEMPTED_REG, as there's no way to tell to GCC that we want to eliminate that static initializer altogether.
It is odd that GCC defined only the enums for the v3 API and didn't implement the feature.
Note the header file is shared among binutils and compilers (GCC, LLVM).
About the get_symbols_v3, apparently it's only used by gold and llvm now:
LLVM: llvm/tools/gold/gold-plugin.cpp
binutils: gold/plugin.cc
Martin, also in addition to skipping resolution file emission, we probably want to take the file away from lto1 command line, so it is not parsed (at least from my quick look at the code today)
Yes, you are correct.
@rui314 There's a GCC LTO plugin version that supports get_symbols_v3:
https://gcc.gnu.org/git/?p=gcc.git;a=shortlog;h=refs/heads/devel/mold-lto-plugin-v2
Can you please experiment with that?
@marxin Sure.
Is there any way to distinguish the plugin with v3 support from the ones without? I have a local patch to restart the linker process to starting over from the beginning to forget about IR files, but I want to do that only when needed. If we can distinguish v2 from v3, we can make mold to execv itself only if the plugin is v2.
I have another question. It looks like gcc linker plugin sometimes adds new symbols _Unwind_Resume and __gxx_personality_v0 as a result of compilation even though they are not added by add_symbols. Is this a bug?
The usual way to figure out the linker plugin version is by seeing what get_symbols_v? variant was called. This was OK with V1->V2 update since the API was backward compatible (it was added on my request to make difference of _IRONLY and _IRONLY_EXP resolutions, since the first allows more optimizations to happen). But it seems that the v2->v3 update is problematic in this respect since you want to know if you need to restart linker before that happen. I am adding Eugeni, Carry and Hj to CC. They may have some ideas.
On Mon, Feb 14, 2022 at 12:22 PM Rui Ueyama @.***> wrote:
@marxin https://github.com/marxin Sure.
Is there any way to distinguish the plugin with v3 support from the one without? I have a local patch to restart the linker process to starting over from the beginning to forget about IR files, but I want to do that only when needed. If we can distinguish v2 from v3, we can make mold to execv itself only if the plugin is v2.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1038942119, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW7IVJL4KZ5H56MVTRTU3DNX3ANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
@marxin I built your gcc branch and try to compile a few programs. mold can be built with your gcc+mold with -flto, which is great. But it cannot built LLVM. It crashes with the following error. I believe you should be able to reproduce the issue with mold at the current git head.
ninja -v lib/libclang.so.15.0.0git
[1/1] : && /home/ruiu/gcc-bin/bin/g++ -fPIC -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wmisleading-indentation -fdiagnostics-color -flto -fno-common -Woverloaded-virtual -fno-strict-aliasing -g -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=mold -Wl,--color-diagnostics -flto -Wl,--version-script,/home/ruiu/llvm-project/clang/tools/libclang/libclang.map -shared -Wl,-soname,libclang.so.13 -o lib/libclang.so.15.0.0git tools/clang/tools/libclang/CMakeFiles/libclang.dir/ARCMigrate.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/BuildSystem.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndex.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexCXX.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexCodeCompletion.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexDiagnostic.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexHigh.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexInclusionStack.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexUSRs.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CIndexer.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXComment.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXCursor.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXIndexDataConsumer.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXCompilationDatabase.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXLoadedDiagnostic.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXSourceLocation.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXStoredDiagnostic.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXString.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/CXType.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/Indexing.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/FatalErrorHandler.cpp.o tools/clang/tools/libclang/CMakeFiles/libclang.dir/Rewrite.cpp.o -Wl,-rpath,"\$ORIGIN/../lib" lib/libclangAST.a lib/libclangBasic.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangIndex.a lib/libclangLex.a lib/libclangRewrite.a lib/libclangSema.a lib/libclangSerialization.a lib/libclangTooling.a lib/libclangARCMigrate.a -ldl lib/libLLVMAArch64CodeGen.a lib/libLLVMAArch64AsmParser.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Disassembler.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMAMDGPUCodeGen.a lib/libLLVMAMDGPUAsmParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUDisassembler.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMCodeGen.a lib/libLLVMARMAsmParser.a lib/libLLVMARMDesc.a lib/libLLVMARMDisassembler.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMAVRCodeGen.a lib/libLLVMAVRAsmParser.a lib/libLLVMAVRDesc.a lib/libLLVMAVRDisassembler.a lib/libLLVMAVRInfo.a lib/libLLVMBPFCodeGen.a lib/libLLVMBPFAsmParser.a lib/libLLVMBPFDesc.a lib/libLLVMBPFDisassembler.a lib/libLLVMBPFInfo.a lib/libLLVMHexagonCodeGen.a lib/libLLVMHexagonAsmParser.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonDisassembler.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiCodeGen.a lib/libLLVMLanaiAsmParser.a lib/libLLVMLanaiDesc.a lib/libLLVMLanaiDisassembler.a lib/libLLVMLanaiInfo.a lib/libLLVMMipsCodeGen.a lib/libLLVMMipsAsmParser.a lib/libLLVMMipsDesc.a lib/libLLVMMipsDisassembler.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430CodeGen.a lib/libLLVMMSP430AsmParser.a lib/libLLVMMSP430Desc.a lib/libLLVMMSP430Disassembler.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMNVPTXDesc.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMPowerPCDesc.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMPowerPCInfo.a lib/libLLVMRISCVCodeGen.a lib/libLLVMRISCVAsmParser.a lib/libLLVMRISCVDesc.a lib/libLLVMRISCVDisassembler.a lib/libLLVMRISCVInfo.a lib/libLLVMSparcCodeGen.a lib/libLLVMSparcAsmParser.a lib/libLLVMSparcDesc.a lib/libLLVMSparcDisassembler.a lib/libLLVMSparcInfo.a lib/libLLVMSystemZCodeGen.a lib/libLLVMSystemZAsmParser.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZDisassembler.a lib/libLLVMSystemZInfo.a lib/libLLVMVECodeGen.a lib/libLLVMVEAsmParser.a lib/libLLVMVEDesc.a lib/libLLVMVEDisassembler.a lib/libLLVMVEInfo.a lib/libLLVMWebAssemblyCodeGen.a lib/libLLVMWebAssemblyAsmParser.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyDisassembler.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMWebAssemblyUtils.a lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a lib/libLLVMXCoreCodeGen.a lib/libLLVMXCoreDesc.a lib/libLLVMXCoreDisassembler.a lib/libLLVMXCoreInfo.a lib/libLLVMCore.a lib/libLLVMSupport.a lib/libclangFormat.a lib/libclangToolingInclusions.a lib/libclangToolingCore.a lib/libclangFrontend.a lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a lib/libclangRewrite.a lib/libclangSerialization.a lib/libclangSema.a lib/libclangEdit.a lib/libclangAnalysis.a lib/libclangASTMatchers.a lib/libclangAST.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMPasses.a lib/libLLVMCoroutines.a lib/libLLVMObjCARCOpts.a lib/libLLVMMIRParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMDesc.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiDesc.a lib/libLLVMLanaiInfo.a lib/libLLVMipo.a lib/libLLVMFrontendOpenMP.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMVectorize.a lib/libLLVMRISCVDesc.a lib/libLLVMRISCVInfo.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZInfo.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMWebAssemblyUtils.a lib/libLLVMGlobalISel.a lib/libLLVMCFGuard.a lib/libLLVMInstrumentation.a lib/libLLVMAsmPrinter.a lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMBitWriter.a lib/libLLVMTarget.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMProfileData.a lib/libLLVMSymbolize.a lib/libLLVMDebugInfoPDB.a lib/libLLVMDebugInfoMSF.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMObject.a lib/libLLVMBitReader.a lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMMCParser.a lib/libLLVMTextAPI.a lib/libLLVMMCDisassembler.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMSupport.a -lrt -ldl -lpthread -lm /usr/lib/x86_64-linux-gnu/libz.so /usr/lib/x86_64-linux-gnu/libtinfo.so lib/libLLVMDemangle.a && :
lto1: internal compiler error: in dwarf2out_register_external_die, at dwarf2out.cc:6072
0x98013b dwarf2out_register_external_die
../../gcc/gcc/dwarf2out.cc:6072
0x87eafd lto_read_decls
../../gcc/gcc/lto/lto-common.cc:1952
0x87f89f lto_file_finalize
../../gcc/gcc/lto/lto-common.cc:2272
0x87f89f lto_create_files_from_ids
../../gcc/gcc/lto/lto-common.cc:2282
0x87f89f lto_file_read
../../gcc/gcc/lto/lto-common.cc:2337
0x87f89f read_cgraph_and_symbols(unsigned int, char const**)
../../gcc/gcc/lto/lto-common.cc:2785
0x867912 lto_main()
../../gcc/gcc/lto/lto.cc:625
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
lto-wrapper: fatal error: /home/ruiu/gcc-bin/bin/g++ returned 1 exit status
compilation terminated.
lto-wrapper failed
_Unwind_Resume and __gxx_personality_v0 is exception handling. I think you will see other symbols added as well (like calls to the libgcc runtime or the hidden partitioned symbols). We always assumed that we can add extra symbols to the binary though this was always somewhat fragile for example with respect of linking together two LTO built blobs (via incremental linking or if some linker will start supporting multiple plugins at once). LLVM never invents any additional symbols?
The dwarf2out ICE seems like GCC bug. It fails on a check for given declaration we have no more than one debug info. I don't think there is linker and resolution info to blame here and it may be side-effect of declaration linking. I will see if I can get it crashing with bfd.
On Mon, Feb 14, 2022 at 12:52 PM Rui Ueyama @.***> wrote:
I have another question. It looks like gcc linker plugin sometimes adds new symbols _Unwind_Resume and __gxx_personality_v0 as a result of compilation even though they are not added by add_symbols. Is this a bug?
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1038960699, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW3RT6SJTCZNZRXJ5J3U3DP4VANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
@marxin I built your gcc branch and try to compile a few programs. mold can be built with your gcc+mold with -flto, which is great.
That are great news!
But it cannot built LLVM. It crashes with the following error. I believe you should be able to reproduce the issue with mold at the current git head.
How do you configure (with cmake) LLVM so that is uses LTO, please?
The usual way to figure out the linker plugin version is by seeing what get_symbols_v? variant was called. This was OK with V1->V2 update since the API was backward compatible
A bit fragile detection can be by seeking for get_symbols_v3 symbol:
$ nm /home/marxin/bin/gcc/libexec/gcc/x86_64-pc-linux-gnu/12.0.1/liblto_plugin.so | grep get_symbols_v3
0000000000015310 b get_symbols_v3
A more robust solution would be a new callback similar to LDPT_GOLD_VERSION, where we can declare plug-in version (that can correspond to GCC version).
get_symbols_v3 is a static variable: static ld_plugin_add_symbols add_symbols, add_symbols_v2; so using it for version detection would break when library gets stripped @.***:~> nm /usr/lib64/bfd-plugins/liblto_plugin.so nm: /usr/lib64/bfd-plugins/liblto_plugin.so: no symbols Adding new callback does not solve the problem with backward compatibility. However it may be a useful thing to have for working around similar problems in the future.
Not sure how justified it is since the plugin API is IMO quite stable and robust - 2 revisions in 10 years, where the first one was very early is not too bad. On the other hand, GCC currently plays quite ugly (and slow) games with copying the early debug sections out of the original object files to temporary object files so linkers understand them. This could be probably solved by a simple plugin API extension.
On Mon, Feb 14, 2022 at 2:51 PM Martin Liška @.***> wrote:
The usual way to figure out the linker plugin version is by seeing what get_symbols_v? variant was called. This was OK with V1->V2 update since the API was backward compatible
A bit fragile detection can be by seeking for get_symbols_v3 symbol:
$ nm /home/marxin/bin/gcc/libexec/gcc/x86_64-pc-linux-gnu/12.0.1/liblto_plugin.so | grep get_symbols_v3 0000000000015310 b get_symbols_v3
A more robust solution would be a new callback similar to LDPT_GOLD_VERSION, where we can declare plug-in version (that can correspond to GCC version).
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1039110330, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW3JBAPVFAFJP7VM3A3U3ECC7ANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: @.***>
@marxin
Here is how I tried to build clang with gcc+mold.
git clone [email protected]:llvm/llvm-project.git
mkdir llvm-project/build
cd llvm-project/build
cmake -GNinja -DLLVM_ENABLE_PROJECTS='clang;lld' -DLLVM_USE_LINKER=mold -DLLVM_ENABLE_LTO=On -DCMAKE_C_COMPILER=/home/ruiu/gcc-bin/bin/gcc -DCMAKE_CXX_COMPILER=/home/ruiu/gcc-bin/bin/g++ ../llvm
ninja
I have installed mold to a $PATH and gcc with your patch to /home/ruiu/gcc-bin.
@janhubicka
It looks like LLVM doesn't add new undefined symbols as far as I tested. I didn't expect that GCC would add a new undefined symbol as a result of compilation because it can result in getting a different output from a LTO build than from a non-LTO build. Adding a new undefined symbol may pull out more object files from an archive, which may define some other symbol that were previously undefined. That means some symbols we have previously reported to the linker plugin via the get_symbols callback as undefined were actually defined symbols. I think that can cause a correctness issue.
I'm testing mold built with commit lto. When building coreutils -fuse-ld=mold -flto mold reports: mold: error: undefined symbol: /tmp/ccKVAs5t.ltrans0.ltrans.o: __udivti3 collect2: error: ld returned 1 exit status make [2]: *** [Makefile: 9626: src/make-prime-list] Error 1 make [2]: *** Waiting for unfinished jobs ....
when I build only with -fuse-ld=mold - build OK Maybe I'm wrong.
@sajcho I think that symbol is another instance of the symbols we discussed above. It's added by LTO after the symbol resolution stage.
Yes it is call to libgcc runtime (128 bit divide). Gcc can definitely indent these as well as calls to libc (memcpy etc)
Dne st 16. 2. 2022 2:48 dop. uživatel Rui Ueyama @.***> napsal:
@sajcho https://github.com/sajcho I think that symbol is another instance of the symbols we discussed above. It's added by LTO after the symbol resolution stage.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1041008058, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW5QGUZT3BXRQNXM34DU3L67VANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
To build llvm-13.0.1 :
cmake ..
-DCMAKE_BUILD_TYPE="Release"
-DCMAKE_INSTALL_PREFIX="/ usr"
-DLLVM_ENABLE_PROJECTS="clang;lld"
-DLLVM_OPTIMIZED_TABLEGEN=ON
-DLLVM_BUILD_LLVM_DYLIB=ON
-DLLVM_LINK_LLVM_DYLIB=ON
-DLLVM_INSTALL_UTILS=ON
-DLLVM_ENABLE_FFI=ON
-DLLVM_ENABLE_ZLIB=ON
-DLLVM_USE_LINKER="mold"
-DLLVM_USE_PERF=ON
-DLLVM_ENABLE_LTO=ON
-DLLVM_ENABLE_EH=ON
-DLLVM_ENABLE_PIC=ON
-DLLVM_ENABLE_RTTI=ON
-DLLVM_INCLUDE_TESTS=OFF
-DLLVM_ENABLE_LIBCXX=OFF
-DLLVM_ENABLE_SPHINX=OFF
-DLLVM_ENABLE_OCAMLDOC=OFF
-DLLVM_INCLUDE_EXAMPLES=OFF
-Wno-dev
../llvm
make -O -j1 V=1 VERBOSE=1
With this configuration, I will get here.
[ 61%] Linking CXX shared library ../../lib/libLTO.so
cd /home/devel/building/work/src/llvm-project-13.0.1.src/build/tools/lto && /usr/bin/cmake -E cmake_link_script CMakeFiles/LTO.dir/link.txt --verbose=1
/usr/bin/g++ -fPIC -O2 -m64 -march=x86-64 -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wmisleading-indentation -fdata-sections -flto -O2 -m64 -march=x86-64 -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword -Wl,-rpath-link,/home/devel/building/work/src/llvm-project-13.0.1.src/build/./lib -Wl,-O3 -Wl,--gc-sections -Wl,--version-script,"/home/devel/building/work/src/llvm-project-13.0.1.src/build/tools/lto/LTO.exports" -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=mold -flto -shared -Wl,-soname,libLTO.so.13 -o ../../lib/libLTO.so.13 CMakeFiles/LTO.dir/LTODisassembler.cpp.o CMakeFiles/LTO.dir/lto.cpp.o -Wl,-rpath,"$ORIGIN/../lib" ../../lib/libLLVM-13.so
mold: error: undefined symbol: /tmp/ccfCmrEe.ltrans0.ltrans.o: vtable for llvm::cl::opt<char, false, llvm::cl::parser
However, the interpretation of the result is beyond my knowledge.
When I use this configuration,
cmake .. -DCMAKE_BUILD_TYPE=Release '-DCMAKE_INSTALL_PREFIX=/usr ' '-DCMAKE_C_FLAGS_RELEASE=-O2 -m64 -march=x86-64 -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword' '-DCMAKE_CXX_FLAGS_RELEASE=-O2 -m64 -march=x86-64 -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword' '-DLLVM_ENABLE_PROJECTS=clang;lld' -DLLVM_USE_LINKER=mold -DLLVM_ENABLE_LTO=ON -DLLVM_INCLUDE_TESTS=OFF -DLLVM_ENABLE_LIBCXX=OFF -DLLVM_ENABLE_SPHINX=OFF -DLLVM_ENABLE_OCAMLDOC=OFF -DLLVM_INCLUDE_EXAMPLES=OFF -Wno-dev ../llvm
-- The C compiler identification is GNU 12.0.1 -- The CXX compiler identification is GNU 12.0.1
llvm builds fine.
The usual way to figure out the linker plugin version is by seeing what get_symbols_v? variant was called. This was OK with V1->V2 update since the API was backward compatible (it was added on my request to make difference of _IRONLY and _IRONLY_EXP resolutions, since the first allows more optimizations to happen). But it seems that the v2->v3 update is problematic in this respect since you want to know if you need to restart linker before that happen. I am adding Eugeni, Carry and Hj to CC. They may have some ideas.
Sorry, the versioning interface is meant for the plugin to tell what version of the plugin API is supported by the linker, not the other way around. But as Honza suggested, you can infer it by noting which version of the get_symbols entry point is called. If that's too late in the process to help, I'm afraid I don't have any better suggestion.
-cary
On Mon, Feb 14, 2022 at 12:22 PM Rui Ueyama @.***> wrote:
@marxin Sure.
Is there any way to distinguish the plugin with v3 support from the one without? I have a local patch to restart the linker process to starting over from the beginning to forget about IR files, but I want to do that only when needed. If we can distinguish v2 from v3, we can make mold to execv itself only if the plugin is v2.
— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>
Yes. I have gcc with patch support LDPT_GET_SYMBOLS_V3 from marxin.
LLVM compiles but is of abnormal size. Maybe it's because the options aren't used LLVM_BUILD_LLVM_DYLIB and LLVM_LINK_LLVM_DYLIB.
I made several changes a few hours ago, so if you rebuild mold and try again, it might pass this time.
- cmake .. -DCMAKE_BUILD_TYPE=Release '-DCMAKE_INSTALL_PREFIX=/usr ' '-DCMAKE_C_FLAGS_RELEASE=-O2 -m64 -march=x86-64 -mtune=generic -fuse-ld=mold -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword' '-DCMAKE_CXX_FLAGS_RELEASE=-O2 -m64 -march=x86-64 -mtune=generic -fuse-ld=mold -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword' '-DLLVM_ENABLE_PROJECTS=clang;lld' -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_FFI=ON -DLLVM_ENABLE_ZLIB=ON -DLLVM_USE_LINKER=mold -DLLVM_ENABLE_LTO=ON -DLLVM_INCLUDE_TESTS=OFF -DLLVM_ENABLE_LIBCXX=OFF -DLLVM_ENABLE_SPHINX=OFF -DLLVM_ENABLE_OCAMLDOC=OFF -DLLVM_INCLUDE_EXAMPLES=OFF -Wno-dev ../llvm -- The C compiler identification is GNU 12.0.1 -- The CXX compiler identification is GNU 12.0.1 ....................................... result:
[ 87%] Linking CXX shared library ../../lib/libLLVM-13.so
cd /home/devel/building/work/src/llvm-project-13.0.1.src/build/tools/llvm-shlib && /usr/bin/cmake -E cmake_link_script CMakeFiles/LLVM.dir/link.txt --verbose=1
/usr/bin/g++ -fPIC -O2 -m64 -march=x86-64 -mtune=generic -fuse-ld=mold -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wmisleading-indentation -fdata-sections -flto -O2 -m64 -march=x86-64 -mtune=generic -fuse-ld=mold -fexceptions -fstack-clash-protection -fcf-protection -D_FORTIFY_SOURCE=2 -Wno-psabi -Wno-address -Wno-missing-template-keyword -Wl,-rpath-link,/home/devel/building/work/src/llvm-project-13.0.1.src/build/./lib -Wl,-O3 -Wl,--gc-sections -Wl,-Bsymbolic-functions -Wl,--as-needed,-z,relro,-z,now -Wl,-z,defs -Wl,-z,nodelete -fuse-ld=mold -flto -shared -Wl,-soname,libLLVM-13.so -o ../../lib/libLLVM-13.so CMakeFiles/LLVM.dir/libllvm.cpp.o -Wl,-rpath,"$ORIGIN/../lib" -Wl,--version-script,/home/devel/building/work/src/llvm-project-13.0.1.src/build/./lib/tools/llvm-shlib/simple_version_script.map -Wl,--whole-archive ../../lib/libLLVMDemangle.a ../../lib/libLLVMSupport.a ../../lib/libLLVMCore.a ../../lib/libLLVMFuzzMutate.a ../../lib/libLLVMFileCheck.a ../../lib/libLLVMInterfaceStub.a ../../lib/libLLVMIRReader.a ../../lib/libLLVMCodeGen.a ../../lib/libLLVMSelectionDAG.a ../../lib/libLLVMAsmPrinter.a ../../lib/libLLVMMIRParser.a ../../lib/libLLVMGlobalISel.a ../../lib/libLLVMBinaryFormat.a ../../lib/libLLVMBitReader.a ../../lib/libLLVMBitWriter.a ../../lib/libLLVMBitstreamReader.a ../../lib/libLLVMDWARFLinker.a ../../lib/libLLVMExtensions.a ../../lib/libLLVMFrontendOpenACC.a ../../lib/libLLVMFrontendOpenMP.a ../../lib/libLLVMTransformUtils.a ../../lib/libLLVMInstrumentation.a ../../lib/libLLVMAggressiveInstCombine.a ../../lib/libLLVMInstCombine.a ../../lib/libLLVMScalarOpts.a ../../lib/libLLVMipo.a ../../lib/libLLVMVectorize.a ../../lib/libLLVMObjCARCOpts.a ../../lib/libLLVMCoroutines.a ../../lib/libLLVMCFGuard.a ../../lib/libLLVMLinker.a ../../lib/libLLVMAnalysis.a ../../lib/libLLVMLTO.a ../../lib/libLLVMMC.a ../../lib/libLLVMMCParser.a ../../lib/libLLVMMCDisassembler.a ../../lib/libLLVMMCA.a ../../lib/libLLVMObject.a ../../lib/libLLVMObjectYAML.a ../../lib/libLLVMOption.a ../../lib/libLLVMRemarks.a ../../lib/libLLVMDebugInfoDWARF.a ../../lib/libLLVMDebugInfoGSYM.a ../../lib/libLLVMDebugInfoMSF.a ../../lib/libLLVMDebugInfoCodeView.a ../../lib/libLLVMDebugInfoPDB.a ../../lib/libLLVMSymbolize.a ../../lib/libLLVMDWP.a ../../lib/libLLVMExecutionEngine.a ../../lib/libLLVMInterpreter.a ../../lib/libLLVMJITLink.a ../../lib/libLLVMMCJIT.a ../../lib/libLLVMOrcJIT.a ../../lib/libLLVMOrcShared.a ../../lib/libLLVMOrcTargetProcess.a ../../lib/libLLVMRuntimeDyld.a ../../lib/libLLVMTarget.a ../../lib/libLLVMAArch64CodeGen.a ../../lib/libLLVMAArch64AsmParser.a ../../lib/libLLVMAArch64Disassembler.a ../../lib/libLLVMAArch64Desc.a ../../lib/libLLVMAArch64Info.a ../../lib/libLLVMAArch64Utils.a ../../lib/libLLVMAMDGPUCodeGen.a ../../lib/libLLVMAMDGPUAsmParser.a ../../lib/libLLVMAMDGPUDisassembler.a ../../lib/libLLVMAMDGPUDesc.a ../../lib/libLLVMAMDGPUInfo.a ../../lib/libLLVMAMDGPUUtils.a ../../lib/libLLVMARMCodeGen.a ../../lib/libLLVMARMAsmParser.a ../../lib/libLLVMARMDisassembler.a ../../lib/libLLVMARMDesc.a ../../lib/libLLVMARMInfo.a ../../lib/libLLVMARMUtils.a ../../lib/libLLVMAVRCodeGen.a ../../lib/libLLVMAVRAsmParser.a ../../lib/libLLVMAVRDisassembler.a ../../lib/libLLVMAVRDesc.a ../../lib/libLLVMAVRInfo.a ../../lib/libLLVMBPFCodeGen.a ../../lib/libLLVMBPFAsmParser.a ../../lib/libLLVMBPFDisassembler.a ../../lib/libLLVMBPFDesc.a ../../lib/libLLVMBPFInfo.a ../../lib/libLLVMHexagonCodeGen.a ../../lib/libLLVMHexagonAsmParser.a ../../lib/libLLVMHexagonDisassembler.a ../../lib/libLLVMHexagonDesc.a ../../lib/libLLVMHexagonInfo.a ../../lib/libLLVMLanaiCodeGen.a ../../lib/libLLVMLanaiAsmParser.a ../../lib/libLLVMLanaiDisassembler.a ../../lib/libLLVMLanaiDesc.a ../../lib/libLLVMLanaiInfo.a ../../lib/libLLVMMipsCodeGen.a ../../lib/libLLVMMipsAsmParser.a ../../lib/libLLVMMipsDisassembler.a ../../lib/libLLVMMipsDesc.a ../../lib/libLLVMMipsInfo.a ../../lib/libLLVMMSP430CodeGen.a ../../lib/libLLVMMSP430Desc.a ../../lib/libLLVMMSP430Info.a ../../lib/libLLVMMSP430AsmParser.a ../../lib/libLLVMMSP430Disassembler.a ../../lib/libLLVMNVPTXCodeGen.a ../../lib/libLLVMNVPTXDesc.a ../../lib/libLLVMNVPTXInfo.a ../../lib/libLLVMPowerPCCodeGen.a ../../lib/libLLVMPowerPCAsmParser.a ../../lib/libLLVMPowerPCDisassembler.a ../../lib/libLLVMPowerPCDesc.a ../../lib/libLLVMPowerPCInfo.a ../../lib/libLLVMRISCVCodeGen.a ../../lib/libLLVMRISCVAsmParser.a ../../lib/libLLVMRISCVDisassembler.a ../../lib/libLLVMRISCVDesc.a ../../lib/libLLVMRISCVInfo.a ../../lib/libLLVMSparcCodeGen.a ../../lib/libLLVMSparcAsmParser.a ../../lib/libLLVMSparcDisassembler.a ../../lib/libLLVMSparcDesc.a ../../lib/libLLVMSparcInfo.a ../../lib/libLLVMSystemZCodeGen.a ../../lib/libLLVMSystemZAsmParser.a ../../lib/libLLVMSystemZDisassembler.a ../../lib/libLLVMSystemZDesc.a ../../lib/libLLVMSystemZInfo.a ../../lib/libLLVMWebAssemblyCodeGen.a ../../lib/libLLVMWebAssemblyAsmParser.a ../../lib/libLLVMWebAssemblyDisassembler.a ../../lib/libLLVMWebAssemblyDesc.a ../../lib/libLLVMWebAssemblyInfo.a ../../lib/libLLVMWebAssemblyUtils.a ../../lib/libLLVMX86CodeGen.a ../../lib/libLLVMX86AsmParser.a ../../lib/libLLVMX86Disassembler.a ../../lib/libLLVMX86Desc.a ../../lib/libLLVMX86Info.a ../../lib/libLLVMXCoreCodeGen.a ../../lib/libLLVMXCoreDisassembler.a ../../lib/libLLVMXCoreDesc.a ../../lib/libLLVMXCoreInfo.a ../../lib/libLLVMAsmParser.a ../../lib/libLLVMLineEditor.a ../../lib/libLLVMProfileData.a ../../lib/libLLVMCoverage.a ../../lib/libLLVMPasses.a ../../lib/libLLVMTextAPI.a ../../lib/libLLVMDlltoolDriver.a ../../lib/libLLVMLibDriver.a ../../lib/libLLVMXRay.a ../../lib/libLLVMWindowsManifest.a -Wl,--no-whole-archive ../../lib/libLLVMExtensions.a ../../lib/libLLVMDebugInfoPDB.a /usr/lib/libffi.so ../../lib/libLLVMExecutionEngine.a ../../lib/libLLVMJITLink.a ../../lib/libLLVMOrcTargetProcess.a ../../lib/libLLVMOrcShared.a ../../lib/libLLVMRuntimeDyld.a ../../lib/libLLVMMIRParser.a ../../lib/libLLVMObjCARCOpts.a ../../lib/libLLVMCoroutines.a ../../lib/libLLVMMSP430Desc.a ../../lib/libLLVMMSP430Info.a ../../lib/libLLVMipo.a ../../lib/libLLVMIRReader.a ../../lib/libLLVMAsmParser.a ../../lib/libLLVMFrontendOpenMP.a ../../lib/libLLVMInstrumentation.a ../../lib/libLLVMLinker.a ../../lib/libLLVMVectorize.a ../../lib/libLLVMGlobalISel.a ../../lib/libLLVMCFGuard.a ../../lib/libLLVMSelectionDAG.a ../../lib/libLLVMAsmPrinter.a ../../lib/libLLVMCodeGen.a ../../lib/libLLVMBitWriter.a ../../lib/libLLVMScalarOpts.a ../../lib/libLLVMAggressiveInstCombine.a ../../lib/libLLVMInstCombine.a ../../lib/libLLVMDebugInfoDWARF.a ../../lib/libLLVMDebugInfoMSF.a ../../lib/libLLVMTransformUtils.a ../../lib/libLLVMTarget.a ../../lib/libLLVMAnalysis.a ../../lib/libLLVMMCDisassembler.a -ledit ../../lib/libLLVMProfileData.a ../../lib/libLLVMOption.a ../../lib/libLLVMObject.a ../../lib/libLLVMBitReader.a ../../lib/libLLVMCore.a ../../lib/libLLVMRemarks.a ../../lib/libLLVMBitstreamReader.a ../../lib/libLLVMMCParser.a ../../lib/libLLVMMC.a ../../lib/libLLVMDebugInfoCodeView.a ../../lib/libLLVMTextAPI.a ../../lib/libLLVMBinaryFormat.a ../../lib/libLLVMSupport.a ../../lib/libLLVMDemangle.a -lrt -ldl -lm /usr/lib/libz.so /usr/lib/libtinfo.so /usr/lib/libxml2.so
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.h:287: warning: type 'llvm::AccelTableKind' violates the C++ One Definition Rule [-Wodr]
287 | enum class AccelTableKind {
|
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/DWARFLinker/DWARFLinker.h:25: note: an enum with different value name is defined in another translation unit
25 | enum class AccelTableKind {
|
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.h:288: note: name 'Default' differs from name 'Apple' defined in another translation unit
288 | Default, ///< Platform default.
|
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/DWARFLinker/DWARFLinker.h:26: note: mismatching definition
26 | Apple, ///< .apple_names, .apple_namespaces, .apple_types, .apple_objc.
|
/home/devel/building/work/src/llvm-project-13.0.1.src/build/lib/Target/AArch64/AArch64GenAsmMatcher.inc:217: warning: type 'SubtargetFeatureBits' violates the C++ One Definition Rule [-Wodr]
217 | enum SubtargetFeatureBits : uint8_t {
|
/home/devel/building/work/src/llvm-project-13.0.1.src/build/lib/Target/AMDGPU/AMDGPUGenAsmMatcher.inc:57: note: an enum with different value name is defined in another translation unit
57 | enum SubtargetFeatureBits : uint8_t {
|
/home/devel/building/work/src/llvm-project-13.0.1.src/build/lib/Target/AArch64/AArch64GenAsmMatcher.inc:218: note: name 'Feature_HasV8_1aBit' differs from name 'Feature_isGFX6Bit' defined in another translation unit
218 | Feature_HasV8_1aBit = 63,
|
/home/devel/building/work/src/llvm-project-13.0.1.src/build/lib/Target/AMDGPU/AMDGPUGenAsmMatcher.inc:58: note: mismatching definition
58 | Feature_isGFX6Bit = 60,
|
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h: In function 'encodeAddend.constprop':
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:99:9: warning: writing 8 bytes into a region of size 4 [-Wstringop-overflow=]
99 | memcpy(LLVM_ASSUME_ALIGNED(
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:247: note: destination object 'buffer' of size 4
247 | alignas(ALIGN) char buffer[sizeof(value_type)];
|
In function 'make_unique',
inlined from 'trim' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:359:10,
inlined from 'trimMitigatedEdges' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:608:31:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'trimMitigatedEdges':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'trim' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:361:10,
inlined from 'trimMitigatedEdges' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:608:31:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'trimMitigatedEdges':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'get' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:332:10,
inlined from 'getGadgetGraph' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:536:77,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:276:79:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'get' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:334:10,
inlined from 'getGadgetGraph' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:536:77,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:276:79:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'get' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:334:10,
inlined from 'getGadgetGraph' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:536:77,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:276:79:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'hardenLoadsWithPlugin' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:628:74,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:314:43:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'hardenLoadsWithPlugin' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:629:70,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:314:43:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'hardenLoadsWithPlugin' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:630:64,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:314:43:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'hardenLoadsWithPlugin' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:631:66,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:314:43:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'trim' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:359:10,
inlined from 'hardenLoadsWithPlugin' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:653:31,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:314:43:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
In function 'make_unique',
inlined from 'trim' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/ImmutableGraph.h:361:10,
inlined from 'hardenLoadsWithPlugin' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:653:31,
inlined from 'runOnMachineFunction' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp:314:43:
/usr/include/c++/12.0.1/bits/unique_ptr.h:998:30: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
998 | { return unique_ptr<_Tp>(new remove_extent_t<_Tp>__num); }
| ^
/usr/include/c++/12.0.1/new: In member function 'runOnMachineFunction':
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/ADT/edit_distance.h: In function 'ComputeEditDistance':
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/ADT/edit_distance.h:65:11: warning: argument 1 value '18446744073709551615' exceeds maximum object size 9223372036854775807 [-Walloc-size-larger-than=]
65 | Row = new unsigned[n + 1];
| ^
/usr/include/c++/12.0.1/new:128:26: note: in a call to allocation function 'operator new []' declared here
128 | _GLIBCXX_NODISCARD void* operator new _GLIBCXX_THROW (std::bad_alloc)
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h: In member function 'resolveAArch64Relocation':
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:99:9: warning: writing 8 bytes into a region of size 4 [-Wstringop-overflow=]
99 | memcpy(LLVM_ASSUME_ALIGNED(
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:247:25: note: destination object 'buffer' of size 4
247 | alignas(ALIGN) char buffer[sizeof(value_type)];
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:99:9: warning: writing 8 bytes into a region of size 4 [-Wstringop-overflow=]
99 | memcpy(LLVM_ASSUME_ALIGNED(
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:247:25: note: destination object 'buffer' of size 4
247 | alignas(ALIGN) char buffer[sizeof(value_type)];
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h: In member function 'resolveBPFRelocation':
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:99:9: warning: writing 8 bytes into a region of size 4 [-Wstringop-overflow=]
99 | memcpy(LLVM_ASSUME_ALIGNED(
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Support/Endian.h:247:25: note: destination object 'buffer' of size 4
247 | alignas(ALIGN) char buffer[sizeof(value_type)];
| ^
In member function '__ct ',
inlined from 'child_end' at /home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Object/Archive.cpp:799:29:
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/include/llvm/Object/Archive.h:147:45: warning: 'MEM <uint16_t> [(const struct Child &)&D.31178 + 40]' is used uninitialized [-Wuninitialized]
147 | ChildFallibleIterator(const Child &C) : C(C) {}
| ^
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Object/Archive.cpp: In member function 'child_end':
/home/devel/building/work/src/llvm-project-13.0.1.src/llvm/lib/Object/Archive.cpp:799:61: note: '
So it's now failing with an internal compiler error. It may be a GCC bug. Even if so, I wonder why only mold triggers that, though.
@janhubicka We want to detect if a given plugin supports v3 before calling all_symbols_read. We can restart the linker after all_symbols_read, but it results in calling all_symbols_read twice, which is very inefficient.
From the API user's point of view, it should be enough if a plugin defines get_symbol_v3_supported dynamic symbol if it supports get_symbols_v3.
Is it too hard for gcc to report all possible undefined symbols early in the claim_file stage rather than after all_symbols_read? From the linker's perspective, it is much easier to guarantee that the the symbol resolution result for LTO is the same as non-LTO build if we are able to know all symbols early. Let me explain that.
Previously, mold implemented LTO in the following steps:
- Read all symbols from all input files
- Resolve all symbols so that we know exactly how each symbol is resolved (this is needed to answer to
get_symbol_v2/v3calls) - Call
all_symbols_readand get an LTO result as an ELF file - Replace existing symbols with new ones read from the LTO output
In the above model, symbol resolution result is fixed in step 2. In step 2, we do not only match undefined symbols but also compute anything that affects the decision as to whether a symbol is exported or not. That includes merging symbol visibility, applying version scripts, applying --execlude-libs, process --export-dynamic, process --exclude-symbols, etc. In step 2, we want to know which files have to be pulled in from archives and which shouldn't, because that also affects symbol export-ness. For each symbol, its visibility is set to the most restricted one; for example, if we have a defined symbol with the global visibility and an undefined symbol with hidden, the result is a defined hidden symbol. Reading a new file could change symbol visibility even if it contains only undefined symbols.
After resolving all symbols, we can answer to the queries from the linker plugin as to how symbols are resolved. In step 4, we simply replaces existing symbols with the one read from the LTO result. We don't need to redo the symbol resolution step.
This model is very robust -- if our symbol resolution result for an LTO build is the same as the one for non-LTO build, we can say that the result will be the same. To put it the other way around, in order to guarantee that our symbol resolution is the same as non-LTO build, we want the linker plugin to give us the exact same set of symbols for an IR file as if it were compiled to a native ELF file.
Now, let me explain what was needed to support a linker plugin that can add new undefined symbols arbitrary as a result of LTO. We need to process them in the following steps:
- Read all symbols from all input files
- Resolve all symbols so that we know exactly how each symbol is resolved (this is needed to answer to
- Call
all_symbols_readand get an LTO result as an ELF file - Read symbols from the LTO result and resolve all symbol again, which may result in pulling in new object files from archive or mark new library as "needed".
With this model, it is hard to guarantee that the symbol resolution result in step 2 is consistent with the one in step 4. Symbol resolution is complicated, so adding just a single undefined symbol could in theory have a massive cascading effect to change how other symbols are resolved. I think this model is too fragile and hard to debug if something goes wrong.
So, I wonder if it's possible to improve gcc so that it reports to us about all possible undefined symbols earlier than it does now? Ideally, for any input file foo.cc, foo.o will contain the exactly same set of defined and undefined symbol regardless of whether the file is compiled to a native ELF file or an IR file. Then it is trivial to guarantee that the symbol resolution results are the same.
Is it too hard for gcc to report all possible undefined symbols early in the claim_file stage rather than after all_symbols_read? From the linker's perspective, it is much easier to guarantee that the the symbol resolution result for LTO is the same as non-LTO build if we are able to know all symbols early.
For runtime calls I think this is not possible (short of claiming every possible runtime symbol just for the case). During the late optimization runtime calls may appear - for example GCC. may detect that a given loop is memcpy or it may pattern match matrix multiplication and dispatch to an optimized library. I remember we hit the problem early on while implementing plugin API and some additional mechanisms were added to handle runtime libraries specially. I do not remember how this was done on the top of my head and have to leave soon. I will check out how this works and also think about it a bit more. You do not want to do this during claim_file, because the outcome of optimization really depends on the knowledge of resolution info.
Note that similarly symbols claimed may be optimized out.
Clang has a bit different organization of the optimization pipeline: (it runs a lot more transforms at compile time and then re-runs them at link-time, while GCC is leaving most of code size expanding optimizations for late optimization since only then one can analyze what parts of the program are hot). So GCC generally benefits more in code size from LTO. For this reason I can imagine to happen less often with clang than with GCC that runtime symbol reference appears late, but I would also expect clang to do similar pattern matching and dispatching to runtime. It may just show less often especially on gigant projects like llvm where most of the symbols are going to be claimed early anyway.
On Fri, Feb 18, 2022 at 9:39 AM Rui Ueyama @.***> wrote:
Is it too hard for gcc to report all possible undefined symbols early in the claim_file stage rather than after all_symbols_read? From the linker's perspective, it is much easier to guarantee that the the symbol resolution result for LTO is the same as non-LTO build if we are able to know all symbols early. Let me explain that.
Previously, mold implemented LTO in the following steps:
- Read all symbols from all input files
- Resolve all symbols so that we know exactly how each symbol is resolved (this is needed to answer to get_symbol_v2/v3 calls)
- Call all_symbols_read and get an LTO result as an ELF file
- Replace existing symbols with new ones read from the LTO output
In the above model, symbol resolution result is fixed in step 2. In step 2, we do not only match undefined symbols but also compute anything that affects the decision as to whether a symbol is exported or not. That includes merging symbol visibility, applying version scripts, applying --execlude-libs, process --export-dynamic, process --exclude-symbols, etc. In step 2, we want to know which files have to be pulled in from archives and which shouldn't, because that also affects symbol export-ness. For each symbol, its visibility is set to the most restricted one; for example, if we have a defined symbol with the global visibility and an undefined symbol with hidden, the result is a defined hidden symbol. Reading a new file could change symbol visibility even if it contains only undefined symbols.
After resolving all symbols, we can answer to the queries from the linker plugin as to how symbols are resolved. In step 4, we simply replaces existing symbols with the one read from the LTO result. We don't need to redo the symbol resolution step.
This model is very robust -- if our symbol resolution result for an LTO build is the same as the one for non-LTO build, we can say that the result will be the same. To put it the other way around, in order to guarantee that our symbol resolution is the same as non-LTO build, we want the linker plugin to give us the exact same set of symbols for an IR file as if it were compiled to a native ELF file.
Now, let me explain what was needed to support a linker plugin that can add new undefined symbols arbitrary as a result of LTO. We need to process them in the following steps:
- Read all symbols from all input files
- Resolve all symbols so that we know exactly how each symbol is resolved (this is needed to answer to
- Call all_symbols_read and get an LTO result as an ELF file
- Read symbols from the LTO result and resolve all symbol again, which may result in pulling in new object files from archive or mark new library as "needed".
With this model, it is hard to guarantee that the symbol resolution result in step 2 is consistent with the one in step 4. Symbol resolution is complicated, so adding just a single undefined symbol could in theory have a massive cascading effect to change how other symbols are resolved. I think this model is too fragile and hard to debug if something goes wrong.
So, I wonder if it's possible to improve gcc so that it reports to us about all possible undefined symbols earlier than it does now? Ideally, for any input file foo.cc, foo.o will contain the exactly same set of defined and undefined symbol regardless of whether the file is compiled to a native ELF file or an IR file. Then it is trivial to guarantee that the symbol resolution results are the same.
— Reply to this email directly, view it on GitHub https://github.com/rui314/mold/issues/181#issuecomment-1044145304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK7OQW6XQ2LCB77BKSAODR3U3YAVRANCNFSM5KSCFIAA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
llvm: Build with mold and without lto requires -fPIC CFLAGS+="-fPIC" CXXFLAGS+="-fPIC" , otherwise, it will result in a crash of R_X86_64_PC32 against. Build using mold, lto, and -fPIC crashes with an internal compiler error.
It looks like I can now build LLVM with the plain gcc11 + mold with LTO enabled. Here is my cmake arguments
cmake -GNinja -DLLVM_ENABLE_PROJECTS='clang;lld' -DLLVM_ENABLE_LTO=On -DCMAKE_C_COMPILER=gcc-11 -DCMAKE_CXX_COMPILER=g++-11 ../llvm
, and I run ninja as mold -run ninja to use mold instead of the default ld.
As per https://github.com/xbmc/xbmc/pull/20891#issuecomment-1050813491 it appears that flto=thin is not working with mold.
I build mold with commit 4caadefac64b99157231e3bb727c2674f8a5d328 now and could use, after this -Wl,--thinlto-jobs=all (LLVM/CLANG) to builds apps without problem. THANK YOU!
About the detection of if get_symbols_v3 is used. I'm suggesting the following extension to the plug-in API that provides a name and compiler version of a linker. Note it's very similar to LDPT_GOLD_VERSION = 2, LDPT_GNU_LD_VERSION.
The current implementation returns GCC 12.0.1.
I'm planning suggesting the patch in next GCC stage1 (after 12.1 gets released):
0001-lto-plugin-add-LDPT_PLUGIN_VERSION.patch.txt
@rui314 What do you think about it?
I think that the string-based version detection is a bit fragile and can cause chaos just like web browser's User-Agent string. I could imagine that other compiler would have to return something like "GCC 12.0.1 (LLVM 20.0)" to make it compatible with older linker plugins.
Maybe a silly question, but why can't we just define a symbol other than onload? What we want is to test whether a plugin supports v3 API or not, so checking the existence of some particular symbol (e.g. dlsym(handler, "supports_v3_api") == nullptr) should satisfy our need. (Which brings us to a bigger question -- why is everything in the plugin API is organized as a callback? It's frankly just hard to use.)
I think that the string-based version detection is a bit fragile and can cause chaos just like web browser's User-Agent string. I could imagine that other compiler would have to return something like "GCC 12.0.1 (LLVM 20.0)" to make it compatible with older linker plugins.
So do you prefer a numeric version similar to GCC_VERSION macro? The version is the identification of the compiler for which the plug-in is done, so GCC 12.0.1 (LLVM 20.0) doesn't make sense.
Maybe a silly question, but why can't we just define a symbol other than
onload? What we want is to test whether a plugin supports v3 API or not, so checking the existence of some particular symbol (e.g.dlsym(handler, "supports_v3_api") == nullptr) should satisfy our need.
I would like to come up with a more generic mechanism and not a special case for support_get_symbols_v3. One can deduce functionality based on version, similarly to what various configure checks do.
(Which brings us to a bigger question -- why is everything in the plugin API is organized as a callback? It's frankly just hard to use.)
Good question, maybe @janhubicka can answer ;)
So do you prefer a numeric version similar to GCC_VERSION macro? The version is the identification of the compiler for which the plug-in is done, so GCC 12.0.1 (LLVM 20.0) doesn't make sense.
I don't want to identify the whole compiler's version but each feature's version. I want to know whether or not a given linker plugin supports version N of feature X, and we are not really interested in what the compiler version is.
Identifying a compiler version number instead of each feature's version has a few problems:
- The linker has to manage which compiler version supports what feature set. Keeping that in sync with newer compilers is laborsome.
- Hypothetically, assume that I create a new C++ compiler which does support LTO. My compiler's plugin can't return "MyNewC++Compiler 1.0" to the linker because no one would recognize that string because it's new and thus all linkers would fall back to the most basic feature set when using my compiler's linker plugin. So I probably would have to return "GCC 12.0.1" (or "GCC 12.0.1 (MyNewC++Compiler 1.0)") instead to force the linker to use new feature set. That's what I said as the web browser's User-Agent problem.
I don't want to identify the whole compiler's version but each feature's version. I want to know whether or not a given linker plugin supports version N of feature X, and we are not really interested in what the compiler version is.
Yes, I understand that. That's pretty reasonable requirement, let me think what we can do about it.
The other thing I'd like to identify is whether I need to close an file descriptor after claim_file_hook or not. It looks like GCC doesn't take the ownership of a given fd while LLVM does, so we close an fd only for GCC to avoid the "too many open files" issue.
The code to close an fd is here: https://github.com/rui314/mold/blob/main/elf/lto.cc#L542
Looking at both bfd and gold, they are really responsible for fd closure:
bfd:
static int
try_claim (bfd *abfd)
{
int claimed = 0;
struct ld_plugin_input_file file;
file.handle = abfd;
if (bfd_plugin_open_input (abfd, &file)
&& current_plugin->claim_file)
{
current_plugin->claim_file (&file, &claimed);
bfd_plugin_close_file_descriptor ((abfd->my_archive != NULL
? abfd : NULL),
file.fd);
}
return claimed;
}
gold:
void
Plugin_manager::cleanup()
{
if (this->any_added_)
{
// If any input files were added, close all the input files.
// This is because the plugin may want to remove them, and on
// Windows you are not allowed to remove an open file.
close_all_descriptors();
}
So there's likely a difference in between GCC and LLVM. Is it problematic for mold? Your code seems reasonable to me.
Yes, it is problematic for mold. There is a workaround, but it's pretty ugly. #362
There is a workaround, but it's pretty ugly. #362
Why is it ugly?
Apparently, the GCC LTO plugin implementation (which is older), does not close the descriptors and the behavior was likely never specified. So it's now implementation-defined and we should preserve it for existing linkers like BFD and GOLD.
It's ugly because it uses the plugin file name to decide whether we should close the fd or not. If a plugin name ends with LLVMgold.so, we don't close the fd because it's LLVM. Otherwise, we close it because it's GCC. But obviously this logic is fragile (there's no guarantee that LLVM always uses that filename for their plugin.)
Well, that brings us back to the suggested tv_plugin_version callback. I know the situation is not ideal, but such detection seems to be reasonable. You can detect the GCC plugin by name and consider others as LLVM plugin. To be honest, I don't expect any new plugin implementation in a near future.
Yeah, right. I still don't like if (LLVM) or if (GCC)-style dispatching, but that's probably practical. Given this situation, I'm OK with your proposal of adding LDPT_PLUGIN_VERSION to the plugin API.
Yeah, right. I still don't like
if (LLVM)orif (GCC)-style dispatching, but that's probably practical. Given this situation, I'm OK with your proposal of addingLDPT_PLUGIN_VERSIONto the plugin API.
Fine, I'll do that once GCC's stage1 opens.
About the undefined symbols problem. I've just tried the current master and I'm able to build e.g. postgresql that was affected by the problem. Have you made mold somehow more robust about it? Or am I just lucky?
I made a change to mold so that after the plugin returns an ELF file to us, we redo symbol resolution from scratch instead of trying to replace IR symbols with the ELF symbols. I think it makes the mold's symbol resolution more robust for LTO. mold can now pull out object files from archives if they are needed as a result of LTO.
I made a change to mold so that after the plugin returns an ELF file to us, we redo symbol resolution from scratch instead of trying to replace IR symbols with the ELF symbols. I think it makes the mold's symbol resolution more robust for LTO. mold can now pull out object files from archives if they are needed as a result of LTO.
Oh great!
One more thing: can you please add auto-loading support for plugins, similarly to what binutils does:
... based plugin it must first
be copied into the ‘${libdir}/bfd-plugins’ directory. All gcc based linker
plugins are backward compatible, so it is sufficient to just copy in the newest
one.
?
For the sake of build reproducibility and cross compilation, we depend only on command line arguments. That is, if you pass the exact same command line arguments and their file contents are the same, we promise we behave exactly the same. For this reason, mold (as well as lld) does not have a notion of implicit default library path or such. All target-dependent knowledge are coded to the compiler and passed from the compiler to the linker. It looks like this policy worked really well, so I don't want to hard-code the plug-in directory to the linker. Can you always pass the latest version of the plugin path to the linker instead?
Oh, you are correct. The autoloading mechanism is actually designed for tools like nm, ranlib or ar.
I've just pushed 2 commits to master that enable plug-in when one configures GCC with --with-ld=which ld.mold:
https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=c083e654bd0f29a365ec957c4c0d4e713fb0b010
There's v2 of the get_version API. I decided for use of an enum and integer version (similar to gold_version).
I'm going to send the patch in next stage1:
0001-LTO-plugin-add-ld_plugin_version-callback.patch.txt