[feature] Dealing with debug symbols in conan-center recipes.

[p-groarke]

What is your suggestion?

Currently, our teams using conan have 2 main reasons why they need to modify conan-center recipes (we are currently copy-pasting, but considering a fork someday). In this issue I'll discuss about our first problem: pdbs.

I understand why conan-center doesn't package pdbs and I am not making a case to add them. I think everybody is well aware how bloated they make packages.

What I'd like to pitch is some sort of builtin way to enable pdbs in our packages, without having to modify conan-center recipes. We host our own artifactory server, yet want to reuse conan-center recipes as much as possible. They are the most compelling "feature" of conan.

This would need to be more than a simple recipe option, as you wouldn't want to update the thousands of recipes on conan-center. I don't know how you'd pull it off tbh. Potentially requiring packages to place pdbs in an expected folder /pdbs? Then, with a conan create toggle including those or not into the final package maybe?

I don't know, but handling pdbs in an elegant way would allow more usage of conan-center as-is, which is a huge win for everyone. Especially with the source backup features, which fulfill security and legal requirements!

ty

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Yes, adding some kind of opt-out for ConanCenter recipes would make sense. I'll have a look to this with the team and see what could be the alternatives, thanks for the suggestion.

[p-groarke]

@memsharded Just to make sure it's clear. We actually want to "opt-in" to pdbs :) Currently, conan-center doesn't package pdbs (as far as I know).

[memsharded]

Just to make sure it's clear. We actually want to "opt-in" to pdbs :) Currently, conan-center doesn't package pdbs (as far as I know).

Yes, with the opt-out, I meant the opt-out of the removal of pdbs, so yes, it would be the same as an opt-in to package the pdbs.

[memsharded]

I have been checking this with the team.

We would like to have first some examples and testing of pdb file usage and consumption. That is, assuming that the pdb files are already packaged, how they are used? How do they locate the sources? (because one of the problems of the pdbs is that they are not relocatable regarding the sources, as they contain absolute paths).

This is important because Microsoft documents 2 different ways:

• /Z7 debug symbols embedded in objects
• /Zi PDB generation

And then recommends:

For distributors of third-party libraries, there is an advantage to not having a .pdb file.

So maybe Conan integrations can also have opt-ins to use Z7, which might be more convenient.

All this can be explored with small proof of concepts to understand the issues better. I'd recommend starting with a simple conan new cmake_lib template, and modify it the minimum to validate the full Debug flow, including the reusage of Debug symbols/pdbs/embedded symbols in consumers using those packages.

[p-groarke]

@memsharded Here's how we use them and package the pdbs.

First RelWithDebInfo, which is arguably the most important since it is a non-negotiable. We don't ship a single product build in Release, everything we ship is RelWithDebInfo (and VS equivalent). If an application crashes, we have a crash report dialog and when it is sent to us, we have a system that parses the stack, using the expected pds (stored internally), and does all its magic. This is essential.

Then Debug, we also package pdbs.

We use /Zi everywhere. Please use /Zi. Hardcoded paths are not an issue as we strip the pdbs. Both a Visual Studio and our crash handler deal with them no problem.

Here is an example of a modified conan-center recipe which gathers the pdbs.

    def package(self):
        save(self, os.path.join(self.package_folder, "licenses", "LICENSE"), self._extract_license())
        cmake = CMake(self)
        cmake.install()
        # the cmake command above should be able to do this but could not find how:
        if self.settings.build_type != "Release":
            src_path=os.path.join(self.build_folder, str(self.settings.build_type))
            dst_path=os.path.join(self.package_folder, "bin")
            copy(self, "*.pdb", src_path, dst_path, keep_path=False)

[fschoenm]

@memsharded

This would be interesting for us too. It would be nice if it could also be somewhat customizable to make it work on Linux. (It's possible to manually separate debug symbols from the binaries and store them in a different location, e.g. with objcopy.)

How do they locate the sources? (because one of the problems of the pdbs is that they are not relocatable regarding the sources, as they contain absolute paths).

There are two ways:

• If the path cannot be found, Visual Studio opens a dialog to ask you for the location of the source on your disk and "recalculates" all paths.
• You can run an indexer on PDB files that stores VCS information for all source files. Visual Studio can then dynamically check out the source code from e.g. a Git repo.

[santhonisz]

Just adding that we use /Zi for our Release and /ZI for our Debug PDB generation for our private packages. Obviously the latter is not suited to Conan Center packages.

We have a post-build step that performs source indexing on the generated PDBs to embed a custom script so that Visual Studio can then fetch the git sources as needed.

Some links for reference:

• Source Indexing is Underused Awesomeness
• Source Indexing - Windows drivers

[p-groarke]

@memsharded Just want to pitch in again. We are about to start something similar for macos and linux. I won't be the one implementing that so I don't have all the details. On both those platforms you have tools to extract debug symbols from executables and libs. Happy to share once we have something working.

So I guess my feature request covers all debug symbols for main OSes / compilers.

[memsharded]

Thanks for the ping.

This is something that we are actively investigating, it goes slowly but we are on it.

[p-groarke]

@memsharded Amazing! Gonna be a game changer for us :)

[juansblanco]

Hi @p-groarke, We just released a new hook that may solve your issue : https://github.com/conan-io/conan-extensions/tree/main/extensions/hooks The hook looks for the PDB files for each DLL in the build folder and copies them next to the DLLs in the package folder so Visual Studio will find them. You can find more information and an example of use here: https://docs.conan.io/2/examples/dev_flow/debug/debugging_visual.html

Note that implementing it as a hook makes modifying the recipes no longer necessary as the hook will execute after the package() method of every recipe. Also note that this is just a starting point and it can be further modified to meet your needs. Let us know if it works for you!

[p-groarke]

@juansblanco Hi there,

Thank you so much for the quick turnaround. A hook seems perfect for this, have a great day!