json-flatfile-datastore icon indicating copy to clipboard operation
json-flatfile-datastore copied to clipboard

Query Async

Open Jaben opened this issue 6 years ago • 4 comments

First of all, thanks for this library.

Quick question: is there support for async querying?

Jaben avatar Apr 26 '19 13:04 Jaben

Hi,

There is no async support for querying. AsQueryable and Find-methods may be slow as those convert collection of JObjects to correct classes, but after conversion is done all objects are in the memory so querying is fast.

Data is in Lazy-object: https://github.com/ttu/json-flatfile-datastore/blob/master/JsonFlatFileDataStore/DataStore.cs#L325

AsQueryable and Find-methods use data in Lazy-obejct: https://github.com/ttu/json-flatfile-datastore/blob/master/JsonFlatFileDataStore/DocumentCollection.cs#L32

We can implement async versions for Find-methods and for AsQueryable if you think those would be useful?

ttu avatar Apr 26 '19 15:04 ttu

For concurrency it would be preferable -- that lock could be async which would help with thread resources. But I don't personally need it for my usage at this time.

Was just curious if I missing something :)

Again, thanks for this useful library!

Jaben avatar Apr 29 '19 18:04 Jaben

Thanks for the feedback!

Have to think how async query-operations would be best to implement. Will write more specifications here when I come up with something.

ttu avatar May 03 '19 12:05 ttu

Maybe this should be implemented with async streams? Syntax might be a little nicer than with Task<IEnumerable<T>>.

Internally both would function in a same way, as first whole JObject would be converted to correct classes. So in the end this is only about syntax.

IDocumentCollection.cs:

Task<IEnumerable<T>> AsQueryableAsync();
Task<IEnumerable<T>> FindAsync(Predicate<T> query);
var itemDynamic = (await store.GetCollection("user")
                            .AsQueryableAsync())
                            .Single(p => p.name == "Phil");

var itemDynamic2 = (await store.GetCollection("user")
		             .FindAsync(p => p.name == "Phil"))
                             .First();
					 
foreach(var item in await store.GetCollection("user").AsQueryableAsync())
{
  // TODO
}

This seems much better option

IDocumentCollection.cs:

IAsyncEnumerable<T> AsQueryableAsync();
IAsyncEnumerable<T> FindAsync(Predicate<T> query);
var itemDynamic = await store.GetCollection("user")
                            .AsQueryableAsync()
                            .Single(p => p.name == "Phil");

await foreach(var item in store.GetCollection("user").AsQueryableAsync())
{
  // TODO
}

ttu avatar Jun 24 '19 20:06 ttu