[Proposal]: Extensions

[333fred]

trafficstars

Discussed in https://github.com/dotnet/csharplang/discussions/5496

^{Originally posted by MadsTorgersen November 30, 2021}

Extensions

• [x] Proposed
• [ ] Prototype: Not Started
• [ ] Implementation: Not Started
• [ ] Specification: https://github.com/dotnet/csharplang/blob/main/proposals/extensions.md

LDM Meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2021/LDM-2021-12-01.md#roles-and-extensions https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-08-31.md#roles https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-09-26.md#roles--extensions https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-02-22.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-12-11.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-02-28.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-06-12.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-06-26.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-07-22.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-09-18.md#extensions-naming https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-09-30.md https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-10-02.md#extensions https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-10-07.md https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-10-09.md https://github.com/dotnet/csharplang/blob/main/meetings/2024/LDM-2024-10-14.md

[hez2010]

I prefer using of or something else to distinguish the type being extended and interfaces:

public extension Foo of int : IA, IB, IC, ...
{
    ...
}

Otherwise it will be too confusing if you are extending an interface:

public extension Foo : IA, IB, IC { }

vs

public extension Foo of IA : IB, IC { }

of can be relatively safely made as a keyword since it's neither a verb nor a noun, so almost nobody would choose of as an identifier.

[HaloFour]

I'm curious as to how the team weighs the relative benefits between "roles" and "extension implementation". It feels that without some additional effort in the runtime the two are somewhat incompatible with each other, so if those differences can't be reconciled which of the features might the team lean towards?

Personally, I find extension implementation much more exciting than roles, but that's just my opinion.

[FaustVX]

@hez2010 Maybe, instead of of, for would be a better name for the keyword as it's already a keyword.

public extension Foo for IA : IB, IC { }

[333fred]

I'm curious as to how the team weighs the relative benefits between "roles" and "extension implementation". It feels that without some additional effort in the runtime the two are somewhat incompatible with each other, so if those differences can't be reconciled which of the features might the team lean towards?

Who gave you an early preview of my notes? They're up now, discussion at https://github.com/dotnet/csharplang/discussions/5500.

[sab39]

Here's a scenario that will be great fun to try to accommodate in the design:

interface IFoo { }
interface IBar { }
class Thing { }
public extension FooThing for Thing : IFoo { }
public extension BarThing for Thing : IBar { }
void Frob<T>(T t) where T : IFoo, IBar { }

Frob(new Thing());

On an unrelated bikeshedding note, what about using the existing reserved keywords explicit and implicit as modifiers, rather than treating roles and extensions as completely separate things? An extension is, more or less, just a role that gets applied implicitly based on type rather than needing to be explicitly named in the declaration. Using implicit role as the syntax would spell that correspondence out more (no pun intended) explicitly?

[CyrusNajmabadi]

@sab39 Given, as you've mentioned, how similar these two concepts are. I too am looking for a good syntactic way to convey that similarity, with a clear way to do indicate in which way they differ. Thanks for the explicit/implicit idea, definitely something we'll consider!

[TahirAhmadov]

I'm not sure if I should re-post my comments from the discussion here? In short, I think the extensions and especially interface "adapters" make sense, but roles don't. The main motivation example - DataObject - is an anti-pattern, IMO; it runs into expensive runtime changes; and causes confusion - now you have to keep in mind when looking at an identifier in a "type receiving context" if it's a type or a role.

Here's a scenario that will be great fun to try to accommodate in the design:

interface IFoo { }
interface IBar { }
class Thing { }
public extension FooThing for Thing : IFoo { }
public extension BarThing for Thing : IBar { }
void Frob<T>(T t) where T : IFoo, IBar { }

Frob(new Thing());

This is complicated, but doable using current constraints of the framework. An anonymous type can be generated:

class <mangled>Thing_IFoo_IBar : IFoo, IBar
{
  internal <mangled>Thing_IFoo_IBar(Thing thing) { this._thing = thing; }
  readonly Thing _thing;

  void IFoo.Foo() { ... } // these member(s) are copied from, or call into, FooThing
  void IBar.Bar() { ... } // these member(s) are copied from, or call into, BarThing
}
Frob(new <mangled>Thing_IFoo_IBar(new Thing()));

The same can be done for generic types, etc. Yes, it's complicated, but unlike roles, it's very possible.

[CyrusNajmabadi]

The main motivation example

This was just one example. It's not the main motivation. We discussed in the LDM that there were definitely plenty of scenarios where you'd still want adapters in a strongly typed way that would be sensible.

[sab39]

@TahirAhmadov That works, more or less, for the specific example I gave, but what if Frob<T> had other constraints like where T : class or where T : struct or where T : new() or where T : SomeBaseClass? What about if it were Frob<T, T2> where T2 : T? In general it's not actually possible to generate an anonymous type that can meet all possible constraints that T would meet and also implement IFoo and IBar. This suggests to me that it's not possible to fully support this scenario without runtime assistance.

[TahirAhmadov]

The main motivation example

This was just one example. It's not the main motivation. We discussed in the LDM that there were definitely plenty of scenarios where you'd still want adapters in a strongly typed way that would be sensible.

If it's not the main motivation, surely it shouldn't be the one discussed in the OP, should it?

[CyrusNajmabadi]

The OP is simply showing a demonstration. This is a broad topic and we need to spend a ton more time on it prior to even getting close to a place where we could write something up that was fully fleshed out and chock full of examples and whatnot.

[TahirAhmadov]

@TahirAhmadov That works, more or less, for the specific example I gave, but what if Frob<T> had other constraints like where T : class or where T : struct or where T : new() or where T : SomeBaseClass? What about if it were Frob<T, T2> where T2 : T? In general it's not actually possible to generate an anonymous type that can meet all possible constraints that T would meet and also implement IFoo and IBar. This suggests to me that it's not possible to fully support this scenario without runtime assistance.

The where T : class constraint simply rejects Thing extensions if Thing doesn't satisfy the constraints. The where T : new() rejects all extensions outright. where T : SomeBaseClass also rejects extensions of Thing because it's a different type. Frob<T, T2> where T2 : T is completely irrelevant.

[BhaaLseN]

Back with .NET Framework, I've often ran into situations where i wanted a Math.Clamp<T>(T value, T min, T max), Math.Max(TimeSpan val1, TimeSpan val2) or Path.GetRelativePath(...) for discoverability; but there was no way for me to get this done. Same with string.Contains(string, StringComparison) etc. except as instance extension (which is somewhat taken care of by extension methods though.) Nowadays most are in there since either .NET Core or .NET 5/6, but it would feel more natural to simply extend the existing classes (with a very high-up namespace inside the project, so its most likely in scope all the time; or with a global using for example) when another one of those situations comes up. At least compare to MathEx, MathUtilities, PathHelpers etc. which often aren't as obvious.

The only thing I don't quite get is why we need two keywords here, role and extension. They mean different things if you talk about them, sure, but does this actually matter once you write the code? I'd assume they'll be lowered to virtually the same thing during compilation after all, and I can practically hear the "what's the difference" question coming when I present this to my team during a knowledge transfer meeting.

[TahirAhmadov]

The OP is simply showing a demonstration. This is a broad topic and we need to spend a ton more time on it prior to even getting close to a place where we could write something up that was fully fleshed out and chock full of examples and whatnot.

That's the thing, it would be very interesting to see an example which would demonstrate how roles make something worthwhile possible or significantly easier, before effort is spent on prototypes, implementation planning, etc.

[CyrusNajmabadi]

That's fine. It's something we're working on at this moment '-). The point was raised and was something we intend to get to and write more on. I def don't want us to get the impression that it's just for that. Thanks!

[orthoxerox]

Roles feel like they need a validator method, something that is invoked to by the "implicit conversion" to ensure that the underlying object can fill in that role. I'm not even sure the conversion should be implicit. I'm sure it will be annoying to do stuff like public Customer Customer => (Customer)this["Customer"]; over and over again, but I also want to be able to say if (payload is Order order) { ....

[sab39]

Hmm, that almost makes it sound like you want Extension DUs...

[orthoxerox]

I don't want them to be a DU per se, it's more similar to getting an object from some API and casting it to the expected type. Right now the roles work more like dynamic instead.

[vladd]

@orthoxerox F# has a feature Partial Active Patterns which looks somewhat like your idea.

[hez2010]

This is complicated, but doable using current constraints of the framework. An anonymous type can be generated:

class <mangled>Thing_IFoo_IBar : IFoo, IBar
{
  internal <mangled>Thing_IFoo_IBar(Thing thing) { this._thing = thing; }
  readonly Thing _thing;

  void IFoo.Foo() { ... } // these member(s) are copied from, or call into, FooThing
  void IBar.Bar() { ... } // these member(s) are copied from, or call into, BarThing
}
Frob(new <mangled>Thing_IFoo_IBar(new Thing()));

The same can be done for generic types, etc. Yes, it's complicated, but unlike roles, it's very possible.

C# isn't the only language on CoreCLR, without runtime support how would you expect roles to be defined and used in other languages? Other languages don't recognize the mangled anonymous class.

[TahirAhmadov]

C# isn't the only language on CoreCLR, without runtime support how would you expect roles to be defined and used in other languages? Other languages don't recognize the mangled anonymous class.

The pseudocode I wrote was specifically for extensions, not roles. In any case, though, the mangled anonymous type is generated at the call site, not at the extension declaration site. Specifically because there can be multiple permutations of extensions (or roles - ignoring the fact that I don't like the idea of roles/shapes), these machinations have to be performed when all the information is available: what interfaces are "adapted", etc. Also, regardless of the language, the extension will have to be added somehow to the metadata; the easiest way would be using a class with some special attributes. The other languages can decide whether to implement this feature - in which case they can interpret these attributes like C# does; otherwise, it becomes a class, probably a static one, which they can use in an "old school way". The same is true for existing extension methods. Further, even when it's a simple scenario, for interface "adaptation" to work, the easiest way is again, an anonymous type:

class Thing { }
interface IFoo { void Foo(); }
extension FooThing: Thing, IFoo { void Foo() { ... } }
void Frob(IFoo foo) { }
// this line:
Frob(new Thing());
// is compiled to this:
class <mangled>Thing_IFoo : IFoo
{
  internal <mangled>Thing_IFoo(Thing thing) { this._thing = thing; }
  readonly Thing _thing;

  void IFoo.Foo() { ... } // these member(s) are copied from, or call into, FooThing
}
Frob(new <mangled>Thing_IFoo(new Thing()));

[Thaina]

I also want to voice that I wish there would be some keyword being reused instead of casting new keyword role and extension. Or at least create only one and use implicit/explicit as above

Or implicit class possible?

Aside from that I have nothing against, and fully support this issue

[hez2010]

instead of casting new keyword role and extension. Or at least create only one and use implicit/explicit as above

Or implicit class possible?

Keywords can be introduced as contextual keywords so it can be made not to introduce breaking changes.

[Thaina]

@hez2010 I know there is no breaking change but it still should be the last option to introduce any new keyword. If there would be any possible for composite or reuse then we should

[333fred]

I found the idea of implicit and explicit very interesting and forwarded your comment to our working group @sab39, thanks for the suggestion!

[iam3yal]

I don't get why roles need an implicit and explicit modifiers, can't they be applied based on the context? what does it mean to have these modifiers? why treat them more or less the same and not exactly the same where the only difference is context? I get the you're trying out different ideas but merge these concepts needs to be core principle the way I think about is similar to aggregation vs composition where aggregation is an extension of existing type and composition is a wrapper the only difference is what they user want them to be based on context and not how they were constructed I don't think we want to end up with a situation where "I can do this when it's explicit but not when it's implicit or vice-versa" but maybe I'm misunderstanding why we need to have different rules for these two concepts, it's not clear whether they are similar or identical features yet but based on the OP I think they are either identical or similar to the point where it can be confusing to grasp why we are speaking about two different concepts.

I'll just copy/paste my comment from the other post so something like this:

// Customer.cs
namespace Data;

public extension Customer : DataObject // Wrapper type
{
    public string Name => this["Name"].AsString();
    public string Address => this["Address"].AsString();
    public IEnumerable<Order> Orders => this["Orders"].AsEnumerable();
}

// JsonDataObject.cs
namespace Data;

using JsonLibrary;

public extension JsonDataObject : DataObject // Extension type
{
    public string ToJson() { … this … }
    public static T FromJson<T>(string json) { … }
}

// Program.cs / Main method
using Data;
using Data.JsonDataObject; // Importing a specific extension type
using Data.*; // Importing all extensions types in the namespace Data

var customer = customer.FromJson<Customer>(args[0]);
WriteLine(customer.ToJson());

[Reinms]

Would this be allowed under roles?

role Foo<T> : T
    where T : ISomeInterface
{
}

or would we be forced to directly extend the interface and bring in boxing conversions all over the place as we implicitly cast back and forth in a generic function?

[TahirAhmadov]

Thinking about it, I imagine this happening:

class Thing { }
interface IFoo { void Foo(); }
// the following line
public extension FooThing: Thing, IFoo { void Foo() { ... } }
// is compiled to:
// these attributes are once per assembly, similar to NRT attributes
class ExtensionTypeAttribute { public ExtensionTypeAttribute(params Type[] types) { ... } ... }
class ExtensionInstanceMemberAttribute {  }
class ExtensionStaticMemberAttribute {  }
// the actual extension becomes:
[ExtensionType(typeof(Thing), typeof(IFoo))]
public static class FooThing
{
  [ExtensionInstanceMember]
  public static void Foo(Thing @this) { ... }
}

void Frob(IFoo foo) { }

// this line:
Frob(new Thing());
// is compiled to this:
class <mangled>Thing_IFoo : IFoo
{
  internal <mangled>Thing_IFoo(Thing thing) { this._thing = thing; }
  readonly Thing _thing;

  void IFoo.Foo() { FooThing.Foo(this._thing); }
}
Frob(new <mangled>Thing_IFoo(new Thing()));

[0x0737]

@eyalalonn

I don't get why roles need an implicit and explicit modifiers...

Why unify roles and extensions? Roles are intended to be used only in specific scenarios, extensions - in pretty much any scenarios. Roles are declared as roles to prevent popping in where it's not appropriate. If you want to apply a role to all conforming types implicitly, you declare an extension for existing role. Extensions are often coupled with the other types in a namespace, so auto-importing them with the rest of the namespace (like existing extension methods) makes sense. Also, extension keyword is more weird for type aliases than role. I think it is better to leave role and extension as they are.

[iam3yal]

@0x0737

Why unify roles and extensions? Roles are intended to be used only in specific scenarios, extensions - in pretty much any scenarios. Roles are declared as roles to prevent popping in where it's not appropriate. If you want to apply a role to all conforming types implicitly, you declare an extension for existing role. Extensions are often coupled with the other types in a namespace, so auto-importing them with the rest of the namespace (like existing extension methods) makes sense. Also, extension keyword is more weird for type aliases than role. I think it is better to leave role and extension as they are.

In the OP Mads stated the following:

Extensions are just roles that apply automatically to their underlying type when brought into scope with a using directive.

In my opinion stating that roles are intended to be used only in specific scenarios is incorrect because like Mads stated above extensions are roles, you're constructing an extension by creating a role, they are just applied differently at least that's what I'm getting at but if this is the case then in my opinion we don't really need more than a single keyword and definitely something like implicit and explicit although they might used optionally to limit the scope or something so when implicit is used then the extension automatically brought into scope but I'm not sure whether this is the intention here.