Proposal: Towards increased adoption

[purkhusid]

The landsacape for .Net in Bazel is pretty scattered at the moment. The following rules are the ones that I know of:

• https://github.com/bazelbuild/rules_dotnet
• https://github.com/Brightspace/rules_csharp
• https://github.com/samhowes/rules_msbuild
• https://github.com/AFASResearch/rules_dotnet (Fork of rules_dotnet)

I met up with @tomdegoede and @hsyed-dojo to discuss how we could help make the .Net story for Bazel a bit more fleshed out and we are all willing to put in the work to make it happen.

The AFASReseach fork for rules_dotnet has dirverged quite a bit from the upstream rules but we think that the best approach for consolidation is to migrate the features that are missing from rules_dotnet from the AFAS fork. These feature include

• Reference assembly support
• More complete NuGet support than the current nuget2bazel tool offers
• A compiler worker implementation for C#

AFAS have also created a BUILD file generator which can be added to rules_dotnet. We want to look into making the BUILD file generator a Gazelle plugin so that we can use all of the great features that Gazelle supports.

The AFAS fork has also diverged in some ways:

• It is no longer using the same launchers that rules_dotnet uses to get rid of the C toolchain dependency
• It was forked before the toolchains were implemented in rules_dotnet

This divergence has to be accounted for when migrating the features.

This repository currently has a single active maintainer(@tomaszstrejczek) so we would like to petition @tomaszstrejczek to add more maintainers to the repository if you agree with this proposal. This would make the migration of features a whole lot faster and efficient.

All 3 of us are willing to help maintain these rules and our companies are actively using them.

• Daníel P. Purkhús (@purkhusid)
• Hassan Syed (@hsyed-dojo)
• Tom de Goede (@tomdegoede)

[purkhusid]

Going to tag some other .Net rule authors in case they have some valuable input for this: @j3parker, @omsmith - https://github.com/Brightspace/rules_csharp @samhowes - https://github.com/samhowes/rules_msbuild

[samhowes]

Thanks for the tag @purkhusid!

rules_msbuild is almost ready for primetime. I started developing it mostly for dotnet build parity and with out of the box ide integration in mind.

My focuses have been on:

1. Simpler nuget management
2. Automated Build file management so users only have to edit their csproj files for project references and nuget packages. This is done with a gazelle language implementation in @rules_msbuild//gazelle/dotnet

I planned on contributing back to https://github.com/bazelbuild/rules_dotnet, however it became clear that the builds that I was looking for were not quite what rules_dotnet was aimed at: rules_msbuild uses msbuild to build the project files, and lets msbuild do the building, without any knowledge of csc. I have not done any performance testing, but this could obviously mean that rules_msbuild could be slower due to the msbuild overhead of executing its own target graph. I've mostly done a lot of work customizing msbuild's execution for caching.

Because I'm using msbuild though, feature parity with dotnet build and ide integration comes mostly for free. Specifically, Grpc was pretty trivial to implement given that grpc-dotnet simply uses a nuget package.

I have not tested f# or vb yet, but everything should work just fine given the approach i've taken.

Given the above, I'm not sure rules_msbuild fits into the approach that rules_dotnet has taken, since rules_dotnet compiles the c# files directly and doesn't use the project files. I'll admit that having two separate rulesets can be confusing for the space though. Happy to hear anyone's thoughts.

[J-Swift]

Just wanted to throw out a 'thank you' to you all for taking the initiative on this, and Tomasz for the stewardship.

[j3parker]

We (Brightspace/rules_csharp) would be happy to have a single implementation. Our focus has been on:

• Go directly to csc, not msbuild (i.e. consider msbuild as something equivalent/parallel to Bazel)
• Have ergonomic BUILD files: they should look like simple csproj files, and should be in principle human maintainable (even if they are auto-generated by tools). See this example or this one.
• Use reference assemblies to improve cache-hit rate.
• BUILD-level support for "InternalsVisibleTo" so that two reference assemblies can be created -- one with the assembly attribute set (e.g. to be consumed by tests,) and one for other assemblies. We believe this is critical to cache hit rates in a large dependency graph (any usage of InternalsVisibleTo creates a larger reference assembly --> lower cache-hit rate, but typically the number of transitive users is much greater than the list of assemblies listed in InternalsVisibleTo). I'm happy to discuss this with anyone, e.g. how this is done without impacting correctness.
• Be equivalent to MSBuild wrt target framework handling. See this file for a hint on our take on this (I don't want to imply it is totally correct.) This is important for a large project with many _binary targets (and 3rd party nuget packages) where we can't necessarily target a consistent framework across the board.
• (At least in the short term) support for building .NET Framework on Linux/Mac (using the nuget package for .NET Framework reference assemblies) (this does not (for us) include running .NET Framework outside of Windows, e.g. with Mono on Linux.)

Although we don't have automated tests for it, our rules were developed by comparing looking out how msbuild invoked csc.exe (in practice and by peeking at its code) and attempting to mirror that exactly.

If there is a clearly usable alternative we would at least be willing to make our implementation private to avoid confusion. It is our impression that none of the rules (including ours) really meet that bar yet.

[samhowes]

Just a small remark on the first comment there with msbuild being equivalent to bazel:

For many of the features of MSBuild I think it is appropriate to think of MSBuild as "bazel-like" and definitely understand the approach of using csc directly.

There is at least some motivation from Microsoft to make MSBuild conducive to "Higher order build systems".

In their docs on "Static Graph" they describe using MSBuild to do "single project isolated builds" which is how rules_msbuild is using MSBuild. They describe an "internal implementation" of a "higher order build system" that uses this mode of execution for MSBuild: https://github.com/dotnet/msbuild/blob/main/documentation/specs/static-graph.md#what-is-static-graph

I imagine their internal implementation is something near their currently in development BuildXL, which appears to be bazel in typescript.

I have no doubt that MSBuild is additional overhead compared to using csc directly though.

[j3parker]

Yeah sorry, I didn't mean to imply that our choice there was some objective truth, just a perspective that guided the design of our rules. The static graph link is nice -- thanks!

[purkhusid]

Thanks for chiming in! I've been closely following the development of @samhowes rules_msbuild and Brightspaces's rules_csharp and there are a lot of great ideas in both implementations. I'll also add that even though I think the community would benefit from having a little cohesion, I dont't think that there is any one-true-way of creating some great Bazel rules for .Net.

I also want to say that the main reason I'm choosing to post this issue here on rules_dotnet and proposing to import some of the great features of the other rules is because of discoverability and familiarity. My feeling is that many newcomers look at the rules under the bazelbuild org as the "official" rules and I think it would benefit the already tiny .Net-in-Bazel community to have the visibility that the bazelbuild org provides.

I've also been following how various other language rules are organizing their rules and I like the recent approach that was taken in rules_nodejs where they decided to keep the core rules pretty slim and then make the addons more independent from the main language rules. A parallel in rules_dotnet could be to have the core rules only about the compiling part and then have other rules for e.g. package management but have the providers part of rules_dotnet for interoperability.

[samhowes]

@purkhusid flattered to have your attention! I would definitely be open to contributing my work back into rules_dotnet in here, and that was my original intention. I moved to a fresh implementation for a little more "space" in my implementation, and for an experimentation space with bazel, as I didn't have much bazel experience when I started out.

Right off the bat, it seems like @rules_msbuild//gazelle/dotnet could be updated to handle rules_dotnet current semantics without much difficulty, and that could be an interesting starting point for integrating.

Happy to talk specifics if anyone is interested.

[tomaszstrejczek]

@purkhusid I don't have permissions to add maintainers. Thereforem I posted to the bazel-dev list the request for adding you, @tomdegoede, @hsyed-dojo as maintainers of rules_dotnet.

[purkhusid]

Thank you very much @tomaszstrejczek. I hope we can be of great assistance and build something awesome on the foundation that you have built so far!

[tomdegoede]

Thanks @tomaszstrejczek I just received the invite!

[purkhusid]

Hey! I've been looking into how we might combine the various features from each .Net ruleset that has been mentioned in this issue.

I would like to mention the most wanted and unique features from each ruleset and then propose a plan on how we might gather them all into one main set of rules.

• https://github.com/bazelbuild/rules_dotnet
  • F# support
• https://github.com/Brightspace/rules_csharp
  • The handling of target frameworks and familiarity of for .Net developers
• https://github.com/AFASResearch/rules_dotnet
  • Solid NuGet support
  • Runfiles loading via STARTUP_HOOK
  • Compiler worker support for C#
• https://github.com/samhowes/rules_msbuild
  • Gazelle plugin to generate BUILD files from .Net project files

I've been looking into what ruleset might be the best one to use as a base and so far I'm liking Brightspace/rules_csharp. The rules are in some ways pretty similar to rules_dotnet but they have more advanced support for framework targetting. I've got a WIP branch where I've added bazel run and bazel test support to the rules: https://github.com/purkhusid/rules_csharp/tree/testing-stuff

I would like to propose a plan to consolidate some of these features:

1. Create a new branch in bazelbuild/rules_dotnet called next
2. Copy Brightspace/rules_csharp to the branch and use as a base
3. Get bazel run working (Already done in https://github.com/purkhusid/rules_csharp/tree/testing-stuff)
4. Get bazel test working (Already done in https://github.com/purkhusid/rules_csharp/tree/testing-stuff)
5. Add proper NuGet support based on e.g. https://github.com/AFASResearch/bazel-dotnet/pull/4/files
6. Add F# support
7. Add worker support for C#
8. Add BUILD file generation from csproj/fsproj files

How does this sound? @tomaszstrejczek @tomdegoede @hsyed-dojo @j3parker @omsmith @samhowes

[j3parker]

Fine with us!

I haven't done an analysis around all the rules, so maybe this is table-stakes now, but one thing that I think you should consider supporting is reference assemblies for better build caching, and the stuff we did around InternalsVisibleTo. If you want I'm available for a quick Zoom/etc. call.

The TL;DR is that csc.exe can produce a "reference-only" DLL suitable for referencing when compiling. This DLL doesn't include method bodies or private things (there is a complication around internal things because of InternalsVisibleTo), so you get better downstream caching (if a library you depend on changes the impl of a function or adds new private helpers then you don't need to re-compile). This can be a significant win in a large project graph. This example can be used to play with/observe the behaviour.

[tomdegoede]

@purkhusid Sounds good! I am just a bit worried about breaking changes to the rules_dotnet repo seeing how diverged everything is. But maybe we should decide to live with that for now. Another approach could be migrating features one by one, maintaining the current rules and slowly making them compatible with the new components. I am not so familiar with the toolchain implementations (I stripped everything in our case :P) but that would seem like a good starting point then.

@j3parker I've looked into that feature and definitely plan to keep it around. We employ ref assemblies already but encounter the same InternalsVisibleTo issue for which you have a nice workaround. Using the multiplex worker however we have been able to live with the inefficiency since it only takes 30 seconds to compile 500 projects. Impoved caching is welcome nonetheless. I also hope we can improve our test-suite caching this way in the future.

I do have some slightly different ideas with regards to multi-targetting than is currently implemented in rules_csharp. Especially in the Nuget resolution story I was hoping to leverage Bazels config_settings and select() statements such that Bazel can resolve framework compatibility. When I find the time I'll try to put a design on paper for that.

[j3parker]

Cool. I have no serious comments RE: multi-targeting other than I believe it is good for it to be ergonomic to the user in BUILD files like with msbuild and as demonstrated by this example BUILD file (which might be a bit "too ergonomic", but it matches what people get with msbuild basically).

Our take on mapping that into Bazel behind the scenes should definitely be taken with a grain of salt.

Some users won't care about this, and small projects can get by with copy+pasting rules/writing loops, but in a large code-base that is actively migrating things it can be pretty important.

[purkhusid]

@j3parker Yeah, I should have mentioned reference assemblies in the feature list above. That feature definitely needs to stay. In my branch I've already got the examples running and the tests working with all the features that were already in your repo except when targeting the old .Net Framework. It's been years since I last used the .Net Framework so I'm going to leave that up to someone else to make it work.

@tomdegoede I think we will have to accept some breaking changes since it would make the work a lot harder to try to migrate each feature and keeping backwards compatibility. I think we can have a notice in the README.md about possible breaking changes until we have a public facing API that we are pretty happy with and then we can release a 1.0 of the rules.

I'm going to find some time later this week or next to get the next branch up and running in this repo since nobody is strongly against this approach. After that I would love to pair with anybody to get the features into the branch.

[oakad]

Regarding the msbuild/csc issue: things likes ASP.NET/Blazor are critical part of modern dotnet ecosystem. Those rely on special code generators invoked by msbuild via custom built-in rules. Moreover, there are plenty of third-party/user developed msbuild extensions addressing various needs. To make things even worse, msbuild supports inline c# code as part of the rule definition and this weird feature is also pretty popular (because "plain" msbuild is rather limited in its capabilities).

I have asked the asp.net guys if there's a possibility for making the code generator interface stable and documented, for other tools to use. And their answer was a pretty firm "no", they consider those an implementation detail of msbuild.

Therefore, a decent implementation of rules_dotnet, capable of supporting modern dotnet projects, must use msbuild. :-)

[tomdegoede]

@oakad I understand your reasoning but I must disagree. To fully benefit from a Bazel build these features need to be re-considered individually for caching, determinism etc. The performance overhead of keeping msbuild in the cycle is simply too expensive for us as well. I don't see a point in keeping two highly configurable build orchestrators in our flow.

[oakad]

It's not about individual preferences, but about the state of dotnet ecosystem. I, personally, haven't seen a single largish dotnet application which was just a bunch of c# files. No such thing in nature. There's always some XAML component, or Blazor, or T4 of some sort. By saying that rules_dotnet are only going to support compilation of c# files and that's it, you're basically saying that rules_dotnet are never going to be useful to any serious dotnet developer.

I, for one, have a practical multi-lingual project with a sizable dotnet component. Upon inspecting the state of various rules_dotnet forks (the present one and AFASResearch mainly) I had to write my own custom rules to invoke a custom tweaked msbuild project with a bunch of ugliest command line arguments. But at least I've got my project going.

[njlr]

My perspective as (primarily) an F# developer is a little different. My ideal scenario would be this:

• Bazel build description acts as ultimate source-of-truth
• Bazel rules do not leverage MS Build
• MS Build functionality like dotnet publish is reimplemented in Bazel rules
• Generate the Nuget packages WORKSPACE section from a paket.lock (for C# projects it probably make sense to use Paket too)
• Ability to generate .sln / .fsproj / .csproj files for IDE integration

However I found a few rough edges in the various rules_dotnet projects that have prevented me for fully adopting Bazel. Currently, I call dotnet build in a pre-build step then Bazel picks up the various bin folders. Of course this doesn't allow Bazel to generate inputs to the MS Build or leverage the cache effectively.

[purkhusid]

I do not agree that .Net Bazel rules would have to wrap msbuild to be useful. I also don't think that it's realistic that these here rules will support every workload that .Net supports but they can be structured in a way that it would be easy to interop (e.g. via providers) with other rules that might have a more specific use case.

@tomdegoede is working on a more robust NuGet implementation for Bazel land hopefully it will land in the next branch in the not so far future.

After we have a solid NuGet support I think we can start looking into adding more functionality that msbuild already has e.g.

• crossgen2
• Blazor
• Single file binaries

I'm also going to point out that we are of course very open to getting contribution for features that you would like to see in rules_dotnet and we can discuss each feature better and how we might want to implement them.

The whole point of this issue is just to try to consolidate some of the great work that has been happening in various forks and then move onwards to making these rules a viable alternative to msbuild.

[Dragnalith]

I am coming after the battle but I would like to support the point of view of @oakad

The .Net ecosystem is more than a C# compiler and nuget package manager.

To make bazel handle all .Net capabilities, it's important to also consider WPF, ASP.Net, .Net MAUI kind of projects and feature like "publish self-contained" and "publish trimmed".

It's not a matter of should we or can we avoid using msbuild. It's a matter of how do we enable the features above in bazel?

I also wish for a granularity where the bazel dotnet rules don't use msbuild, but is it possible? In the .Net ecosystem Msbuild is more than just a build orchestrator a lot of build action are implemented in msbuild language (i.e xml, inline .cs) and the stable interface is the .csproj file.

[samhowes]

Do you mean https://github.com/samhowes/rules_msbuild?

[Dragnalith]

@samhowes I am not sure if it is a question for me, but if it is:

Yes I think your rule_msbuild is the most realistic approach to support the dotnet ecosystem in bazel.

PS: I am quite impressed by your work. It relies a lot on undocumented msbuild knowledge, I wonder how you did it.

[oakad]

@samhowes I, personally, would give your project a try - I already need a go helper tool and csproj patching to build my stuff, and your project takes it all to a bona fide next level.

These rules assume you have used dotnet tool install -g samhowes.bzl to set up your workspace and run bazel run > //:gazelle after adding any source files, nuget packages, or project references.

Will be so much better if it was somehow possible to avoid pre-installing a dotnet tool. Experience shows, that any additional requirement on less obvious preinstalled dependencies opens a cornucopia of user issues. :-)

[peakschris]

Hi all, @samhowes @purkhusid

I have been getting rules_dotnet running on our large codebase. We have around 1000 targets successfully building. However, a smaller number based on net48 and involving some complexities such as WPF won't work. This impacts around 20 targets.

So what to do? I've briefly tried rules_msbuild, but am facing some issues with it needing a custom gazelle and not working with dotnet.exe version 7 tooling. The custom gazelle seems to be due to a gazelle MR not having been accepted since 2021. Even if I could get it to work, I'd prefer to use a native bazel engine for the 95% of targets that rules_dotnet supports.

Using both rules_msbuild and rules_dotnet together is a bit messy because they have different nuget integrations.

My suggestion is that we incorporate a cut-down version of the msbuild_library/binary targets into rules_dotnet, to handle those situations where msbuild is required.

Any thoughts?

Chris

[purkhusid]

I think it will be easier to get WPF working rather than having msbuild rules because it's very likely that there will be some hard to solve clashes between msbuild/Bazel. If someone wants to take a stab at adding some msbuild_* rules that interop with the csharp_*/fsharp_* then I think I would not be opposed to having them in here as an escape hatch.

I saw your issue (#405) regarding .Net 4.8 and that is supported, we just don't have an example. My bandwidth is quite limited at the moment but if you could do a PR that adds an WPF example without the Bazel bits then I could look into making it work in Bazel.

[peakschris]

@purkhusid thank you! I've created a repo with two WPF examples in it. If you could have a look at getting it working that would be great. https://github.com/peakschris/rules_dotnet_wpf. Is this OK?

If needed I will work on collecting together a stripped down msbuild_library/msbuild_binary rule for our codebase and will happily contribute that. I believe others may have similar already - can anyone share any starting points for me? Obviously I can already see the rules_msbuild stuff.

[purkhusid]

Closing this issue since I feel like it has served it's purpose. Feel free to open new issues for e.g. WPF support or an MSBuild rule as an escape hatch.