Support SelectMany() overload with transform

[rwasef1830]

Given this class:

class User
{
    public Guid Id { get; set; }
    public List<string> Logins { get; set; }
}

This query

session.Query<User>
    .SelectMany(
        x => x.Logins,
        (u, l) => new
        {
            UserId = u.Id,
            Login = l        
        });

Generates a transforming CTE that selects the Logins collection only, while the final transformation assumes it will have an "Id" property.

~~Will make a PR asap.~~ Looks like some kind of surgery is needed.

[mysticmind]

@rwasef1830 Marten does not have support for directly getting projected results using Select or SelectMany but for getting the whole documents. You have to use .ToList() and then run the SelectMany(...) Can you please change your code as below and let me know your results:

session.Query<User>()
    .ToList()
    .SelectMany(
        x => x.Logins,
        (u, l) => new
        {
            UserId = u.Id,
            Login = l
        });

[rwasef1830]

I know that would fix it, but I was hoping to be highly optimized and avoid fetching the large user object. I worked around it with raw sql query, but it's nice if this can be supported some day.

I dug in the linq provider, the problem is that marten doesn't track individually where each property assigned in the projection comes from and it relies on covering layers of CTEs all returning a single column 'data'. Adding the support seems to be not difficult, but understanding the linq provider model to be able to do it is, and I unfortunately don't have enough free time to do this kind of surgery now.

[mysticmind]

@rwasef1830 I agree with you that getting just the required columns optimizes the fetch on large documents. We are aware about the lack of fetching projected documents, it is yet to be looked at.

[rwasef1830]

I'll leave this as it maybe of use to future implementers (or myself in the future):

The solution relies on detecting references to the parent context, and always having each CTE return 2 columns... The new data and also the data output of the previous CTE... When building then next step use the correct column alias when referencing the values in the new projection.

[jeremydmiller]

@rwasef1830 That's gonna be nasty to implement. Someday post 4.0 maybe.

[jeremydmiller]

Getting deeper into this one today. I don't think this is going to be easy to support at all. Might be related to theoretical, future GroupJoin work