What is the expected workflow for Symbols Validation of official releases?

[ericstj]

Description

We were looking at the results of https://dev.azure.com/dnceng/internal/_build?definitionId=1303&_a=summary and see many failures.

Who's meant to review these and file issues or follow up on the failures? @michalpavelka @mmitche

We noticed many symbol failures but it's hard to know if those are real problems or issues with the pipeline.

Reproduction Steps

Examine failures in https://dev.azure.com/dnceng/internal/_build?definitionId=1303&_a=summary

Expected behavior

Most release builds are green - for any failures issues are filed.

Actual behavior

Many failing validation checks, unclear if they are actionable failures or not.

Regression?

???

Known Workarounds

No response

Configuration

No response

Other information

No response

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @dotnet/runtime-infrastructure See info in area-owners.md if you want to be subscribed.

[ericstj]

@tommcdon @hoyosjs - can you help triage this? Does this belong in runtime or should it go somewhere else? I'm not sure where this responsibility lies.

[hoyosjs]

The validation itself is part of the staging/release pipeline - that'd be on Prague's ProdCon plate with our release team here. If the infra makes the right validation but we don't pass it, this is definitely something for repo owners to go triage. Things get complex fast (what repo does r2r vs not, how do they ship symbols, etc). But a simple "all packages should have a symbol for all entries, unless explicitly in the symbol exclusion list".

[tommcdon]

I agree with Juan - validation should be part of the pipeline that publishes the symbol, which is the staging and release pipelines. As Juan points out, we sometimes index multiple symbols to one shipping binary. These specialized requirements are documented on https://github.com/dotnet/diagnostics/blob/main/documentation/symbols/README.md. Our team (.net runtime diagnostics) should be able to work with the staging/release pipeline owners to ensure that those requirements are being validated.

[ericstj]

Who needs to take action on this to fix it? Can you tag them?

[michalpavelka]

@ericstj We'll look into this and let you know.

[pavel-purma]

In .NET Release Infrastructure team in Prague we have completed this: https://github.com/dotnet/release/issues/1132 It moved all the implementation about symbols publishing validation tests to C# code in our pipeline locally. No other Azure Functions dependencies needed anymore. We hope this will bring us more stability to the symbols validation step.

From next release these steps will start acting in the release process

Are there any specific packages/ symbol files we should take a look more closely as they could be these not publishes properly and maybe check previous runs of versions and validate symbols once more for them?

[hoyosjs]

A good subset of libraries, at least winforms, and then one of the runtime packs from runtime since it has interesting native assets.