Attribute controlling IQueryable mapping

[trejjam]

Is your feature request related to a problem? Please describe. It is valid to write the following:

dbContext.MyEntity
  .Select(x=> new MyDto
  {
    MyEnum = ConvertEnum(x.DbEnum),
  })
  .ToListAsync()

MyEnum ConvertEnum(DbEnum dbEnum) => (MyEnum) dbEnum;

as long as there is no database operation after (e.g. .Select(...).Where(...))

Describe the solution you'd like A new attribute, that will control whether other mapping functions may be called or not. Something like: [QueryableMapping(EQueryableMapping.AllowHydratationBeforeMapping)] and [QueryableMapping(EQueryableMapping.DatabaseServerOnly)] (default)

Additional context Right now, the only functionality that is blocked is ByName enum mapping. But for example, custom mappings or other mapping are allowed for other types.

The idea is with AllowHydratationBeforeMapping to allow ByName enum mapping with a mapping function. With DatabaseServerOnly disable the use of custom mappers and other mappings.

The issue with AllowHydratationBeforeMapping is, that the whole entity is loaded and the mapping is called after. One of the advantages is that EF custom converters are executed. With DatabaseServerOnly big entities with mapping to Dto { public in Id { get; set; } } are effective (no need to transfer throw-away properties).

[latonz]

I don't really understand whats the impact of this attribute. Can you probably come up with some generated code samples for each of the settings?

[trejjam]

Sure:

DTOs:

class A
{
    public B Value { get; set; }
}

class B
{
    public int Value { get; set; }
}

class C
{
    public D Value { get; set; }
}

class D
{
    public float Value { get; set; }
}

DatabaseServerOnly:

//mapping:
partial class Mapping
{
    [QueryableMapping(EQueryableMapping.DatabaseServerOnly)]
    public partial IQueryable<C> MapToC(this IQueryable<A> queryable);

    private D MatToD(B value)
    {
        return new D(value.Value);
    }
}

//generated:
partial class Mapping
{
    public partial IQueryable<C> MapToC(this IQueryable<A> queryable)
    {
        return queryable.Select(
            x => new C 
            {
                Value = new D
                {
                    Value = (float)x.Value.Value,
                }
            }
        );
    }
}
//diagnostics:
// MatToD can not be used inside MapToC

//possible usage:
dbContext.A.MapToC()
  .Where(x=>x.Value.Value > 0)
  .ToListAsync();

AllowHydratationBeforeMapping:

//mapping:
partial class Mapping
{
    [QueryableMapping(EQueryableMapping.AllowHydratationBeforeMapping)]
    public partial IQueryable<C> MapToC(this IQueryable<A> queryable);

    private D MatToD(B value)
    {
        return new D(value.Value);
    }
}

//generated:
partial class Mapping
{
    public partial IQueryable<C> MapToC(this IQueryable<A> queryable)
    {
        return queryable.Select(
            x => new C 
            {
                Value = MatToD(x.Value),
            }
        );
    }
}

//not supported usage:
dbContext.A.MapToC()
  .Where(x=>x.Value.Value > 0)
  .ToListAsync();

// MapToC must be the last LINQ chain which needs to be translated to DB
// following should be still possible
dbContext.A.MapToC()
  .Select(x=>x.Value)
  .ToListAsync();

the advantages of AllowHydratationBeforeMapping are:

• mapping can be more complicated utilizing user-defined mappings
• EF core converters are used

---

disclaimer: I know that the naming of the attribute and values is not great and it can be better, I do not see the better one now

[latonz]

I missed this notification, sorry... Thank you for the explanation. The current implementation isn't dependent on a database or similar (it only uses IQueryable) therefore I'm not sure if DatabaseServerOnly would be a good fit here. Also the hydration seems to be kind of EF Core related, isn't it? An alternative implementation could be based on the proposed IgnoreMappingMethodAttribute (https://github.com/riok/mapperly/issues/891). A property IgnoredMappingTypes or similar of type enum with bitmasked values None, All, Ordinary and QueryableProjection could be added to the IgnoreMappingMethodAttribute. That would also work for your use-case, wouldn't it?

In the past I thought about a mechanism to inlining simple user-implemented mapping methods into the queryable select expression. Would this also address this issue?

[trejjam]

Also the hydration seems to be kind of EF Core related

That's possible, I have no experience with different engines using IQueryable.

An alternative implementation could be based on the proposed IgnoreMappingMethodAttribute

I am not sure how should Both, and Ordinal should work.

QueryableProjection can be part of the IgnoreMappingMethodAttribute, but my goal was that a specific Projection can use mapping methods (generated / user-defined). Maybe IncludeMappingMethodsAttribute(Include=IncludeMappingTypes.None | UserDefined | Generated | All=(UserDefined | Generated)) with None as default - resulting in fully translatable query.

e.g.

record A(E1 Enum, int Value);
record B(E2 Enum, int Value);

[Mapper]
static class Mapper
{
  [IncludeMappingMethods(Include=None)] // can be ommited, None is default
  public static partial IQueryable<B> ProjectToB(this IQueryable<A> queryable);

  [IncludeMappingMethods(Include=UserDefined)]
  public static partial IQueryable<B> ProjectToBWithLocalMapMethods(this IQueryable<A> queryable);

  [MapByValue]
  private static E2 MapToE2(E1 source);

  // non-sense mapping only to demonstrate in projection
  private static int MapWithScale(int source) => int * 100;
}

generates:

[Mapper]
static class Mapper
{
  public static partial IQueryable<B> ProjectToB(this IQueryable<A> queryable) => queryable.Select(
    x=> new B(
      (E2) x.Enum,
      x.Value
    )
  );

  public static partial IQueryable<B> ProjectToBWithLocalMapMethods(this IQueryable<A> queryable) => queryable.Select(
    x=> new B(
      MapToE2(x.Enum),
      MapWithScale(x.Value)
    )
  );
}

[latonz]

@trejjam I meant Ordinary instead of Ordinal, I updated my comment. The idea of IgnoreMappingMethodAttribute is to exclude user implemented mappings from invocation by Mapperly. The idea of the additional property would be to exclude them only from regular mappings (Ordinary) or from queryable mappings (QueryableProjections) or both. This way mapping methods could be excluded from queryable projection mappings.

But why not using two mappers in the first place? This would also solve this, wouldn't it?

In the past I thought about a mechanism to inlining simple user-implemented mapping methods into the queryable select expression. Would this also address this issue? In your example this would lead to

public static partial IQueryable<B> ProjectToBWithLocalMapMethods(this IQueryable<A> queryable) => queryable.Select(
    x=> new B(
      (E2) x.Enum,
      x.Value * 100
    )
  );

[trejjam]

I understand the meaning of IgnoreMappingMethodAttribute.

The issue: Right now, in Mapperly there is an artificial condition that prevents using ByName enum mappings instead of calling the mapping method MapToE2.

The idea of preventing of calling other map methods is useful in some cases, in others, it's not. I see a benefit in the possibility of controlling how the ProjectTo mapping is constructed (for which use). At the same time mapperly does not allow the use of MapToE2, but it uses other user-defined mappings in projections. What should it do, forbid user mappings or use them? For that, I would like to have a controlling mechanism (an attribute).

A side effect of using other mappings inside ProjectTo is that this chain works as expected:

• database column: Type: "E_LEGACY_TYPE_AS_STRING"
• EF core converter: value => value switch { "E_LEGACY_TYPE_AS_STRING" => MyEnum.Value, }
• mapperly projects from entity to dto and from entity enums to dto enums.

Here comes the issue. When Mapperly calls (E2) x.Enum what it really does is executing query SELECT CAST (string_enum_value AS int) AS DtoEnum. Which produces nonsense.

When you write queryable.Select(x=>MapToEnum(x.Value)) it first executes EF core converters, after that the data has a meaning, and after it calls mapping at C# runtime.

It's true, you can write two mappers, but you still have a problem choosing which strategy should be used.

[latonz]

Sorry if I still don't understand, perhaps my understanding of the EF Core query mechanism is too limited. You want that the mapping gets executed on the client side, is that correct? But isn't that exactly what you usually don't want with queryable projection mappings? Couldn't that be simply implemented with Mapperly like this: queryable.AsEnumerable().Select(x => MapperlyMapper.Map(x)) without any Mapperly side queryable projection mappings involved? Can you probably show me a code example how it would look like if you would write it completely manually?

[trejjam]

I want only a necessary part to be executed on the client. 😃

Database entity:

class Entity
{
  public int Id { get;set; }

  public MyEnum Type { get;set; }

  public string LongDescription { get;set; }

  // a lot of other properties
}

Dto:

class Dto
{
  public int Id { get;set; }

  public MyDtoEnum Type { get;set; }
}

Usage:

context.Entities
  .Select(
    x => new Dto
    {
        Id = x.Id,
        Type = MapEnum(x.Type),
    }
  )
  .ToListAsync();

static MyDtoEnum MapEnum(MyEnum value) => /* enum mapping executed on the client */;

This will execute something like this:

SELECT Id, Type FROM database_table; // notice that there are not all Entity properties

If I should write it without EF MAGICK

context.Entities
  .Select(
    x => new
    {
        Id = x.Id,
        Type = x.Type,
    }
  )
  .AsEnumerable()
  .Select(x => {
     // This is not an expression, and it is not transpiled to SQL
     var type = x.Type switch
     {
       A => B,
     };

     return new Dto
     {
        Id = x.Id,
        Type = type,
    };
  })
  .ToListAsync();

I can do it without the first select, but that will result in loading throwaway data from the SQL server.

---

I am not an EF guru, but I tried this, and it works. I also saw it as part of some talk about EF (I believe it was ASP.NET Comunity Standup). The speaker was part of/close to the EF dev team, so it is not a side effect that will disappear with the next release.

---

When I have mapper:

class Entity
{
  public DateTime Created { get;set; }
}

class Dto
{
  public DateTimeOffset Created { get;set; }
}
partial class Mapper
{
  public partial IQueryable<Dto> Project(IQueryable<Entity> queryable); 

  public DateTimeOffset MapToDateTimeOffset(DateTime dateTime) => new (dateTime, TimeSpan.Zero);
}

Mapperly generates:

partial class Mapper
{
  public partial IQueryable<Dto> Project(IQueryable<Entity> queryable) => queryable.Select(
    x=> new Dto{
      Created  = MapToDateTimeOffset(x.Created),
    }
  );
}

Which is what I am searching for to have for enums also.

At the same time, I know it may not be what you always need. e.g. when you want to filter on projected IQueryable. But our codebase contains exactly one place that needs to utilize this type of pure projection.

[latonz]

Oh wow, EF Core does more magic than I knew 🙈 With IgnoreMappingMethodAttribute, you could control which mapping methods are used by queryable mappings. With an additional fix, which would just generate ordinary mapping methods for code, which is not supported in expression trees your use-case would be addressed, without introducing EF Core specifics in Mapperly, wouldn't it? IMO Mapperly should just contain no specifics to EF Core (or as few as possible).

[github-actions[bot]]

:tada: This issue has been resolved in version 3.5.0-next.3 :tada:

The release is available on:

• GitHub release
• v3.5.0-next.3

Your semantic-release bot :package::rocket:

[latonz]

3.5.0-next.3 tries to inline all expressions, see the updated docs. If you want to opt out of inlining for a given method just use a method body block ({}) instead of an expression (=>). Feedback is welcome.