Version compatibility across our packages

[gregkalapos]

This is an issue to discuss version compatibility across our packages.

An example that can easily happen:

• Elastic.Apm is at version [X] and then in version [X+1] we change the signature of an existing API (either remove it, add to it, or change one of the parameters). We release version [X+1] of all our packages.
• In this case when we have dependencies across all our packages, those depend on version [X+1] (e.g. Elastic.Apm.AspNetCore v [X+1] depends on Elastic.Apm` v [X+1]).
• The Elastic.Apm package is used in a library of a user and the library is updated to version [X+1].
• Some of our other package is also used by the same user (e.g. Elastic.Apm.AspNetCore) in an application. This application references version [X] (or earlier).
• The user update the library to use version [X+1] of Elastic.Apm, but does not update the dependent package in the application.
• In this case calls from the dependent package (e.g. Elastic.Apm.AspNetCore) with version [X] will try to call into Elastic.Apm with a method signature of version [X], which changed causing MissingMethodException.

Tasks in this issue:

• [ ] Decide how far we want to support these scenarios (allow in minor/major? Nor allow et all? Only support matching versions? Something else?).
• [ ] Document the decision.
• [ ] See what others do in this scenario.
• [ ] Create a strategy and implementation around testing it. If we support this, we'll need have a whole matrix and run those in CI.

[Matiszak]

From my perspective it would be very desirable (from developer perspective) to have as much breaking-free behaviour. For example recently I was upgrading a bunch of projects from .NET5 to .NET6. We have so many internal libraries that retargeting them would be a tedious task for all of them. Instead I checked all of the breaking changes of libraries from Microsoft and as far as I remember I found maybe 1-2 breaking changes very deep in technical packages that I imagine we would never use directly (low level networking stuff). And that was for major version upgrade.

From perspective of my company we'd like to switch to Kibana APM but to do so we need to be as much breaking-free as possible (take for example Dynatrace which as far as I know are profiler only and I never encountered failure scenario there except for one performance problem).

I'd suggest maybe to evaluate if sticking to guidelines defined https://docs.microsoft.com/en-us/dotnet/core/compatibility/#modifications-to-the-public-contract will be acceptable for your goal of APM agents. I'm sure it would build more confidence in the software (and trust me, overall score of how problematic are different solutions of tracing is a huge factor in adoption across company).

[gregkalapos]

Summery of a discussion from the weekly meeting:

https://github.com/nullean/assembly-differ seems to be an ideal tool to test for such things. We could add it as part of the CI pipeline and it'd let us know if we break something. The check works based on comparing public API method signatures.