csharplang
csharplang copied to clipboard
dotnet
Proposal: extend the set of things allowed in expression trees
Expression trees support only a subset of the things that could be semantically meaningful. Consider allowing them to support the following constructs:
- The null-coalescing operators such as a?.b, a?[b]
- Expressions involving
dynamic - Invocations that use default or named parameters
- Assignment and other mutation expressions
- A statement-bodied lambda
I think allowing the null operators in expression trees would be one of the most useful, especially in regards to frameworks that provide LINQ support for databases.
I believe that supporting statement-bodied lambdas would make expression trees a lot more interesting in many real world scenarios. I posted to the Visual Studio UserVoice site a while back, and it seemed like there is some interest in this.
+1 for enabling ?. in expression trees. I did a pull request some months ago (https://roslyn.codeplex.com/SourceControl/network/forks/Olmo/NullPropagationInExpression/contribution/7700) and discuses the idea (https://roslyn.codeplex.com/discussions/571077). It's not complicated but tell me if you're interested in and updated version.
Also Invocations that use default or named parameters in expression trees will be nice :)
I don't think that dynamic would be possible, because it is handled runtime, which would prevent the ability for the expressions to be handled during compilation.
I am also very much interested in allowing statement-body like @VictorBlomberg for dsl purposes. It would open some interesting scenarios in manipulating expression trees that right now are a pain.
We now have many different ways to represent code as an object model - CodeDom, LINQ Expression Trees, and now Roslyn. Since Roslyn is the most complete of all the models (and will continue to be so) is there anyway it could become the basis for lambda expressions going forward? Perhaps by introducing something like a RoslynExpression<TDelegate> (name not important) in order to support backward comparability by not changing Expression<TDelegate> (and ecosystem). This way there isn't a continuous need to create parallel representations of lambda expressions to support new language features.
That said, a couple things jump out as making this a less ideal approach:
- New model based on Roslyn syntax trees would not be language agnostic
- Differences between C# and VB syntax trees would make consumption a little more involved
- Existing code that uses Expression<TDelegate> and ecosystem would require much more rework to take advantage of the new model
Thoughts?
@iSynaptic I wouldn't consider the CodeDOM object model for any new code. It doesn't support even some of the most basic things (like static classes).
And regarding Roslyn and Expression trees, I would actually like to see expression trees expand to be able to do more things (e.g. that you could generate whole types using them), not the other way around, like you're proposing. I think that Expression trees and Roslyn syntax trees have very different primary use cases and because of that there are significant differences between them. And those differences make the kind of metaprogramming you want to do with Expression trees much harder to do with Roslyn.
Some of the differences include:
- Roslyn can precisely encode any C#/VB source, including whitespace, comments and invalid code, which makes it much more complex
- Roslyn has separate syntax and semantic layers and neither works with types that you can manipulate at runtime (like
TypeorMethodInfo) - Roslyn is language-specific, Expression trees work the same in C#, VB and F#
I probably think the Expression Trees have the following huge weaknesses:
- Not reclaimable by GC. Expression.Compile() and friends tend to eat up your memory, thus a special caching logic is needed all the time;
- Not suitable for writing types -- only static methods are supported. This is probably pain number one. Even if you could compile them into Type methods, the generated methods are still static. This prevents proper proxying of some classes (think Castle.DynamicProxy and friends on steroids). The result is having custom boilerplate to handle class state. There is a whole host of problems that could be solved by expression trees type generation. Think for example:
- Code that optimizes itself on-the-fly based on configuration parameters;
- Custom language parsers (in combination with ANTLR and friends);
- Dynamic code generation (generic interface implementation, handy for, say, network communication).
@hivanov On the other hand great thing about Expressions: it's currently the only way to run "dynamic code" (without writing own interpreter) inside Windows Store Applications as as far as I understand correctly they are even .NET Native compatible. If tendency is to go towards AOT instead of JIT: dynamic types are probably not coming back.
Regarding GC issues: I am using a lot of Expression.Compile and never had memory issues with them (maybe due to the caching logic) - I must check this out.
And of course: +1 for "null-coalescing operators".
@mkosieradzki: Regarding System.Linq.Expressions.Expression in .NET Native: Yes, as far as I'm aware, they are supported, albeit not in a compiled way (no runtime code generation is possible), but in a slow, interpreted way.
@axel-habermaier Turns out that the interpreted way is actually faster than the compiled way if the expression is executed less than ~50 times.
It's not slow. It's good-enough for scenarios I am using it for (scripting inside application).
@hivanov I am unable to reproduce memory leak behavior when using Compile. I have tried to create millions of different expressions, evaluate them and have literally no memory leaking. I would expect at least one "byte" leaking per expression but after GC memory returns to the initial usage. Are you sure there is a memory leak issue related to Expressions?
@mkosieradzki It actually depends on the expressions you build. I have found out, on numerous occasions, that expressions that reference external stuff (mostly, mapped external variables) fail to be released even if the referencing code + the variables are no longer reachable. Especially if the variables are something like channel description.
I still stand behind my general idea for the extension of the Expression Trees to be able to generate whole classes or, at least, methods that support "this", so they could be used in TypeBuilders.
+1 for statement bodied lambdas.
My use case for this has always been the ability to write GPU programs against a framework that just compile as expressions in C# and then get hoisted and transpiled to a GPU specific byte code at runtime such as nVidia's CUDA PTX or even higher level stuff like DirectX/OpenGL's shaders or OpenCL via some kind of provider model.
:+1: for all these features. I really love expression trees and it would be nice to have more support from the language.
It could also be nice to have something like [ReflectedDefinition] attribute (similar to F#) to get expression tree from method. Imagine having a ORM which will save the method to DB and allow using it in linq queries.
@exyi this is actually a really good idea. If you can expose Properties and Methods as expression trees you can factor-out a lot of complexity of LINQ queries in reusable parts.
We use a cumbersome convention involving declaring the method body as an static Expression<T> field, an ExpressionFieldAttribute and some MsBuild / Cecil magic to bind those together.
https://github.com/signumsoftware/framework/commit/a7e369966d99dc8a6fc36eb7a0b7227d50223f30
public class PersonEntity
{
(...)
//How it is today
static Expression<Func<PersonEntity,bool>> IsAmericanExpression = p=>p.Country == "USA";
[ExpressionField]
public bool IsAmerican
{
get { return IsAmericanExpression.Evaluate(this); }
}
//How it could be
[ExpressionDefinition]
public bool IsAmerican => this.Country == "USA";
}
@gafter If I do a pull request for this, any chance it will get accepted?
@olmobrutall +1. We have the same problem. I think that the scenario you are describing is very common in a lot of application that are, for example, Entity Framework based where one want to build a set of expression and reuse them by combining them.
@olmobrutall It's quite often good idea to steal F# features :).
I wanted it to use in DotVVM framework to translate simple functions to javascript. It would be quite similar what FunScript do, since they use F#'s ReflectedDefinition attribute.
@gafter Isn't it planned to make Expression classes a record type so the following would be possible?
switch(expr) {
case LambdaExpression(UnaryExpression(MemberExpression memberExpr)):
case LambdaExpression(MemberExpression memberExpr):
...
}
// instead of
switch(expr) {
case LambdaExpression { Body is UnaryExpression { Operand is MemberExpression memberExpr } }:
case LambdaExpression { Body is MemberExpression memberExpr }:
...
}
// (I'm using OR patterns, don't mind)
Also, is there any ExpressionType like enum generated for record types as an optimization for pattern matching (like Tag in F#) or it's just type checking?
Should the pure and impure expressions be differentiated in C#? May be that will enable some enhancements in the cases of pure expressions. Or this has been done already?
+1 for more Expression Trees support, especially the null-propagating accessors or statement-bodied lambdas (as far as I know the latter are already possible, except it requires using troublesome Expression Trees API to build them). Wouldn't like these to lag too far behind the existing C# features.
On a side note, would it be possible to support async/await with LINQ expressions as well, or is it too much of a stretch? According to this SO thread it'd require a major compiler rewrite, but I'd like to have some official C# devs stance to refer to. ^^'
Think about the problem conceptually. The await keyword actually generates some syntax precompile which generates a state machine (You can find various levels of detail for this online, but here - for example: http://www.filipekberg.se/2013/01/16/what-does-async-await-generate/). It tracks things like
- starting
- completed
You wouldn't have an expression so much as a complex set of imperative code expressed using symbols which make it seem functional. It would DEFINITELY require a major compiler rewrite. It would also inject lots of complexity to your code and make it very difficult to debug.
@bartdesmet has done a bunch of prototype work seemingly related to this issue: https://github.com/bartdesmet/ExpressionFutures/tree/master/CSharpExpressions
One thing that I've always wished that I could do with expression trees in C# was to be able to define a helper or extension method that could be invoked from within the expression but instead of a MethodCallExpression being emitted a helper method would be invoked that would expand into a portion of the expression tree.
:spaghetti:
[ExpressionExpansion(MethodName = nameof(IsBetweenExpression))]
public static bool IsBetween(this int value, int minimum, int maximum) {
return (value >= minimum) && (value <= maximum);
}
public static Expression IsBetweenExpression(Expression value, Expression minimum, Expression maximum) {
return Expression.AndAlso(
Expression.GreaterThanOrEqualTo(value, minimum),
Expression.LessThanOrEqualTo(value, maximum)
);
}
@HaloFour With #8990 it will resolve into a pattern. So you don't need to worry about the MethodCallExpression itself.
@alrz
I don't see how that proposal is related. I wouldn't be interpreting these expression trees myself, I'd be passing them to some provider like Entity Framework. Unless you're suggesting that I wrap EF and translate the expression trees prior to it then interpreting them.