[FEATURE] Source, SourceIncludes & SourceExcludes to support lambda targeting types & list

[TheFireCookie]

Is your feature request related to a problem? Please describe. I'm frustrated upgrading from 7 to 8 because in the include/exclude & general source filtering in NEST, we could work with lambda pointing to properties of a specific document (mydoc => mydoc.MyField), in the new ES8 client, we can only work with list of strings and the mapping is not easy.

NEST:

gd => gd.SourceIncludes(doc=> doc.Id, doc=> doc.CreationDate)

Describe the solution you'd like Handling of the list (in addition of array etc) on the Fields class

var esIncludeFields = new List<string> { "id", "type" };
new SearchRequestDescriptor<TDoc>()
.SourceIncludes(esIncludeFields)

Handling of the .NET lambda targeting type properties

new SearchRequestDescriptor<TDoc>()
.SourceIncludes(doc => doc.Id, doc => doc.CreationDate)

Describe alternatives you've considered Right now I'm using workaround of course :)

[nchetvertukha]

@TheFireCookie Hi! Can you please specify the workaround that you use, please? That would be very useful for me considering moving to this version, despite the fact that all documentation about it had been written for previous versions. Thanks!

[TheFireCookie]

For now I've just written those 2 extensions methods

public static class SourceIncludesExtensions
{
  public static SearchRequestDescriptor<TDocument> SourceIncludes<TDocument>(this SearchRequestDescriptor<TDocument> searchRequestDescriptor, params Expression<Func<TDocument, object>>[] sourceSelectors) where TDocument : class, new()
  {
    var sources = sourceSelectors.Select(e => (Field)e).ToArray();
    searchRequestDescriptor.SourceIncludes(sources);
    return searchRequestDescriptor;
  }
  
  public static GetRequestDescriptor<TDocument> SourceIncludes<TDocument>(this GetRequestDescriptor<TDocument> searchRequestDescriptor, params Expression<Func<TDocument, object>>[] sourceSelectors) where TDocument : class, new()
  {
    var sources = sourceSelectors.Select(e => (Field)e).ToArray();
    searchRequestDescriptor.SourceIncludes(sources);
    return searchRequestDescriptor;
  }
}

[flobernd]

Just want to mention the "official" way of achieving this:

var sd = new SearchRequestDescriptor<MyDocument>();
sd.SourceIncludes(new Expression<Func<MyDocument, object>>[] {
    x => x.Id,
    x => x.name
});

However, I agree that a direct overload might be a good idea and therefore leaving this issue open as a reminder.

[nchetvertukha]

Thanks a lot!

[TheFireCookie]

Just want to mention the "official" way of achieving this:

var sd = new SearchRequestDescriptor<MyDocument>();
sd.SourceIncludes(new Expression<Func<MyDocument, object>>[] {
    x => x.Id,
    x => x.name
});

However, I agree that a direct overload might be a good idea and therefore leaving this issue open as a reminder.

The "official" seems too noisy for no added benefits for my taste but I didn't know we could write it like that, thanks for the heads up!

[jade-lucas]

Just want to mention the "official" way of achieving this:

var sd = new SearchRequestDescriptor<MyDocument>();
sd.SourceIncludes(new Expression<Func<MyDocument, object>>[] {
    x => x.Id,
    x => x.name
});

However, I agree that a direct overload might be a good idea and therefore leaving this issue open as a reminder.

Would be great if we could get this example out on the official documentation. I have been finding it difficult to migrate NEST 7.XX to Elastic.Clients.ElasticSearch 8.XXX.

[TheFireCookie]

Just want to mention the "official" way of achieving this:

var sd = new SearchRequestDescriptor<MyDocument>();
sd.SourceIncludes(new Expression<Func<MyDocument, object>>[] {
    x => x.Id,
    x => x.name
});

However, I agree that a direct overload might be a good idea and therefore leaving this issue open as a reminder.

Would be great if we could get this example out on the official documentation. I have been finding it difficult to migrate NEST 7.XX to Elastic.Clients.ElasticSearch 8.XXX.

If you have specific questions, you can ask me if you want. I've finished migrating my project :)

[flobernd]

Feel free to drop me specific questions as well. Improving the documentation is high up on my todo list and I'm already collecting all kinds of useful code snippets that will end up in the docs at some point.

[MEmanuelsson]

I agree with @jade-lucas comment, I'm having problems grasping the new way of "talking to Elasticsearch" compared to how this worked in Nest. Maybe I've missed some kind of basic explanation of the architecture in the new client? The code snippets you talk about @flobernd would probably be very helpful to get access to. Any idea about when these will be available?

An example of a query I want to migrate to the new client, a MatchAll-query that in Nest looks like this (fieldNames being a list of strings):

await Client.SearchAsync<T>(sd => sd.Index(indexAlias).Query(q => q.MatchAll()).Size(10000).Source(sfd => sfd.Includes(descriptor => descriptor.Fields(fieldNames.ToArray()))));

How do I translate this to using the new client?

[TheFireCookie]

I agree with @jade-lucas comment, I'm having problems grasping the new way of "talking to Elasticsearch" compared to how this worked in Nest. Maybe I've missed some kind of basic explanation of the architecture in the new client? The code snippets you talk about @flobernd would probably be very helpful to get access to. Any idea about when these will be available?

An example of a query I want to migrate to the new client, a MatchAll-query that in Nest looks like this (fieldNames being a list of strings):

await Client.SearchAsync<T>(sd => sd.Index(indexAlias).Query(q => q.MatchAll()).Size(10000).Source(sfd => sfd.Includes(descriptor => descriptor.Fields(fieldNames.ToArray()))));

How do I translate this to using the new client?

    var fieldNames = new List<string> { "abc", "def" };
    var indexAlias = "my-index";
    await elasticsearchClient.SearchAsync<T>(sd => sd.Index(indexAlias).Query(q => q.MatchAll()).Size(10000).SourceIncludes(fieldNames.ToArray()));