[MEDI] Allow Pipeline and Reader to work with any Source

[adamsitnik]

The idea is to introduce a new interface, called IIngestionDocumentReader, where the generic type parameter specifies the source. Source can be anything (FileInfo, Stream but also int or Guid or custom user type). It's up to the reader to get the document for given source (parse a file or read it from DB or somewhere else).

public interface IIngestionDocumentReader<TSource>
{
    Task<IngestionDocument> ReadAsync(TSource source, string identifier, string? mediaType = null, CancellationToken cancellationToken = default);
}

The IngestionDocumentReader class remains, we are still opinionated and believe that readers should implement FileInfo and Stream support. It implements the new interface. But we also know that it's not enough for all the scenarios, so Pipline no longer accepts the old reader class, but something that implements the new interface.

Because of that, the pipeline now needs to specify two generic type arguments instead of one. This is a breaking change. Moreover, pipeline itself no longer defines methods for processing multiple files or directory. This functionality was moved to extension methods.

fixes #7082

Microsoft Reviewers: Open in CodeFlow