Track XDP main

[ProjectsByJackHe]

Description

Let's track XDP main instead of a release branch as currently MsQuic has dependencies on stuff outside of the release branches.

Testing

CI

Documentation

N/A

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 85.60%. Comparing base (b0ce510) to head (b8b9edb). :warning: Report is 17 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5638      +/-   ##
==========================================
- Coverage   86.13%   85.60%   -0.54%     
==========================================
  Files          60       60              
  Lines       18663    18663              
==========================================
- Hits        16076    15976     -100     
- Misses       2587     2687     +100     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[nibanks]

I only caution that at msquic release branches only reference xdp release branches.

[mtfriesen]

I only caution that at msquic release branches only reference xdp release branches.

This is a good point - @ProjectsByJackHe can you update the release docs to make changing the submodule to the latest XDP release branch part of the release steps? Though if we do that, we in turn could make releasing MsQuic into a refactoring exercise if the most recent XDP release branch is missing compile-time dependencies. Hmm.

[guhetier]

I only caution that at msquic release branches only reference xdp release branches.

This is a good point - @ProjectsByJackHe can you update the release docs to make changing the submodule to the latest XDP release branch part of the release steps? Though if we do that, we in turn could make releasing MsQuic into a refactoring exercise if the most recent XDP release branch is missing compile-time dependencies. Hmm.

I think having to change the submodule when creating a release branch is somewhat error prone / easy to forget.

If there are difference in the main and xdp release branch, it might mean we introduce a bug in the release branch that was not happening in main.

If we need to update the submodule by hand from time to time, I'd rather have to do it in main, rather than at release time.

[ProjectsByJackHe]

I only caution that at msquic release branches only reference xdp release branches.

This is a good point - @ProjectsByJackHe can you update the release docs to make changing the submodule to the latest XDP release branch part of the release steps? Though if we do that, we in turn could make releasing MsQuic into a refactoring exercise if the most recent XDP release branch is missing compile-time dependencies. Hmm.

I updated the release docs. I think to avoid the refactoring hassle in MsQuic whenever you create a new release, if there is no release branch that integrates nicely with this new MsQuic release, devs should make a new XDP release first.

That way, active development in main for MsQuic tracks the latest features in XDP, and releases for MsQuic track releases in XDP, which I think should be the way to go.

[guhetier]

That way, active development in main for MsQuic tracks the latest features in XDP, and releases for MsQuic track releases in XDP, which I think should be the way to go.

I am not sure I agree with that. MsQuic devs and XDP devs happen to be largely the same group, but it should not be the responsibility of MsQuic release process to figure out when to create an XDP release or if a specific build is stable.

[mtfriesen]

Agreed with all points above: XDP and MsQuic releases must be decoupled, but MsQuic must also avoid taking fragile dependencies on unreleased XDP code within its release branches.

I've been thinking about this a bit more, and maybe MsQuic needs to take build dependencies two different ways:

1. The official/released SDK nuget provided by XDP matching the official XDP runtime package. This will be the only mode supported by official/release builds, and needs to also be part of the main test matrix. This allows MsQuic + XDP to be built and tested only against officially supported artifacts.
2. MsQuic can also track prerelease/main code, either continuing to use the submodule on some non-release commit, or by taking a parallel SDK nuget + runtime nuget dependency, with both tracking prerelease code. Since most XDP features require runtime nuget changes, it implies XDP must create a prerelease for any cross-project engineering, so there's no additional cost to publishing the prerelease SDK nuget. Alternatively, MsQuic could use the XDP submodule to consume both the compile time and build its runtime test dependencies.

[nibanks]

Many of these problems are why we had decided to always have official MsQuic branches referenced official releases of XDP. I still feel pretty strongly that should stay the case. It's fine if there is a way for private branches to take temporary dependencies on unreleased XDP (or other dependencies), but merging that into main causes problems...

[ProjectsByJackHe]

Many of these problems are why we had decided to always have official MsQuic branches referenced official releases of XDP. I still feel pretty strongly that should stay the case. It's fine if there is a way for private branches to take temporary dependencies on unreleased XDP (or other dependencies), but merging that into main causes problems...

If we want to go back to this model, we need to wait until XDP releases v1.2, update submodules to point to v1.2, and make sure future devs don't check in MsQuic code that depends on some unreleased feature in XDP main into MsQuic main. I think this is the easiest way forward for now.

[mtfriesen]

To be clear, the problem isn't to determine what is easiest for us now, but what is the best (balancing safe, reliable, efficient, agile) way to maintain the dependency between these two projects.

We've discovered tracking the latest XDP release branch keeps MsQuic a little too far behind, but we're now in a bad state where MsQuic is ahead of release but still supposedly tracking the release branch.

I generally agree with both Nick and Guillaume that we need to track this dependency in a more robust and predictable way, including, but not limited to, we must depend only on released features in release branches.