Remove variable group - Dotnet-Blob-feed

[epananth]

• [ ] This issue is blocking
• [ ] This issue is causing unreasonable pain

Dotnet-Blob-feed is no longer used in Arcade. But the product repos still reference it in their pipelines, so it's time to remove this variable group so that we don't have to maintain it.

Engineering services team is going to remove this variable group by 08/20/2023, so product teams please remove the variable group and fix your builds.

cc: @mmitche @riarenas @ilyas1974

Release Note Category

• [ ] Feature changes/additions
• [ ] Bug fixes
• [ ] Internal Infrastructure Improvements

Release Note Description

[dougbu]

@epananth has this been communicated to our partners❔ or, are you searching product team pipelines for UI and YAML mentions of the variable group❔

[epananth]

Not yet, I will this week.. I will have to change the date on this.. I kinda lost track of this after I was done with SSA last time. I will complete it this time..

[chcosta]

Sent mail to partners with a removal date of 11/1/2023

[chcosta]

In the interest of not putting anyone in a completely blocked / super impactful state, my recommendation would be to actually revoke all pipeline permissions on 11/1, and then wait until later before actually deleting the variable group. Revoking the pipeline permissions will put repos in a blocked state ("Permissions needed"). This will notify pipeline owners (assuming they look at their pipeline) and allow us to unblock them while also taking note of the dependency. I actually noticed a pipeline we own which references the variable group.

Next steps:

• 11/1: revoke all pipeline permissions to the variable group
• Future, delete variable group

[chcosta]

After 11/1/2023, we will be removing pipeline access to the DotNet-Blob-Feed variable group. If your pipeline references the variable group, you may see a message like this when you try to run a build…

DotNet-Blob-Feed references were removed from Arcade in Remove vars and steps that shouldn't be necessary by mmitche · Pull Request #11577 · dotnet/arcade (github.com) which corresponds to Arcade Sdk version 8.0.0-beta.22564.8 (Arcade commit: 8df5191cc3062d2bdb0d24a7e26e6c87f9f96a9d)

If you are on an older version of Arcade, you probably still aren't relying on the DotNet-Blob-Feed variable group, but you may still have a reference to it in your yaml templates.

Mitigation steps:

• If you have a reference to DotNet-Blob-Feed in your code base (not under eng\common, then you can probably directly remove it safely (see the note below for a caveat).

  • Example removal - Commit 443f5bd5: Remove dotnet-blob-feed - Repos (azure.com)

• If you have a reference in your source code to "DotNet-Blob-Feed" under the eng\common folder, then you are probably on an older version of Arcade.

  • If you can take an updated version of Arcade, please do so.
  • If your code base is locked, then you can simply remove references to DotNet-Blob-Feed
    • Note: If you ever move to a newer version of Arcade, you may end up with a file conflict against any files which you directly modify under eng/common

Notes:

• All repos should be using publishing v3. With publishing v3 enabled, the "DotNet-Blob-Feed variable is simply not used. If you're not using publishing v3, please reach out to dotnetes.
  • Publishing v3 documentation - arcade/Documentation/CorePackages/Publishing.md at main · dotnet/arcade (github.com)

[riarenas]

Permissions have been removed from the variable group. I estimate we can delete this variable group at any point after 12/1 (but we might want to wait until next year)

[riarenas]

This variable group is used by both roslyn and F# builds. @MilenaHristova gave those permissions back, but that means this group cannot be removed yet. Throwing this back to the backlog

[MilenaHristova]

This variable group is used by both roslyn and F# builds. @MilenaHristova gave those permissions back, but that means this group cannot be removed yet. Throwing this back to the backlog

O no, I'm sorry. I should've told them to remove the dependency to the variable group?

[riarenas]

Not at all, I think you did the right thing by unblocking them! I'll just mention to examine the removal after I do some digging. Roslyn and F# have some special circumstances on how they take arcade updates.

[riarenas]

It just looks like they have references to the variable groups in their yaml, and not in an arcade template. I'll let the teams know to look at the remediation comment.

[riarenas]

This group is also... used by arcade-official. I just granted permissions once again.

[riarenas]

Add dotnet-interactive to the list of repos still using this.

[dkurepa]

The variable group wasn't removed on arcade 6.0. This is causing issues for the https://github.com/dotnet/install-scripts repos, which is using arcade 6.0.

[dkurepa]

In general, don't we need to remove it on 6.0 and 7.0 for servicing? @epananth

[riarenas]

Yep... I think this issue still needs work and investigation before we can remove permissions again. At the very least this variable group should be removed from every arcade template on every arcade branch.

[MilenaHristova]

dotnetfeed-storage-access-key-1 is used in dotnet release pipeline, information in https://github.com/dotnet/release/issues/709

[missymessa]

Current status: The variable group has been removed from release/7.0, release/8.0, main branches of Arcade.

We have some issues with removing this from release/6.0, so we'll chat about it in triage.

[missymessa]

Ensure that Secret Manager no longer requests for the unused secret to be rotated.

[garath]

Ensure that Secret Manager no longer requests for the unused secret to be rotated.

And then close this issue. We got the primary win of removing this group from Arcade main and release/8, and the tail of getting the remaining bits extracted are proving troublesome.

[premun]

Update from our side: I moved arcade-services onto 8.0 Arcade last week so we can remove the usage. @andriipatsula will take care of that.

[premun]

dnceng-shared is in the same boat as arcade-services. I believe the CI was copied from arcade-services when being created. Seems like dnceng-shared could also use a commit like this one: https://dev.azure.com/dnceng/internal/_git/dotnet-wpf-int/pullrequest/35321?_a=files https://dev.azure.com/dnceng/internal/_build/results?buildId=2308725&view=results

Its internal pipeline is stuck on waiting on permissions too but it will need a bump from 6.0 to 8.0 Arcade too. Volunteers? (I won't be able this week)

[dkurepa]

The arcade 6.0 validation broke on the PR that removed the variable group. I opened up a PR to revert it since we need to get https://github.com/dotnet/arcade/pull/14248 in. Revert PR: https://github.com/dotnet/arcade/pull/14257