json-flatfile-datastore icon indicating copy to clipboard operation
json-flatfile-datastore copied to clipboard

Updated GetNextIdValue to get actual next Id

Open qpooqp opened this issue 6 years ago • 5 comments

Potential fix for issue #3.

I'm taking all id values and then picking the max value from them.

qpooqp avatar Jun 18 '18 19:06 qpooqp

Would you like to add a benchmark that adds e.g. 1k, 10k, 100k etc. items to collection? With that we could compare how much this will slow down inserting new items.

I can't see any other solution for this so if this slows down insert a lot, maybe we have to add some limit e.g. for collections under 1000 items we go through all items and for over 1000 items just take last item's id.

ttu avatar Jun 20 '18 06:06 ttu

I tried to make some benchmarks, but the performance was worse than i thought. 😄 I made some improvements, but i need to polish it a little bit. Give me some time and i will commit the code.

qpooqp avatar Jun 21 '18 17:06 qpooqp

There is some ugly solutions to try e.g.

  • Serialize and find id value from the text or JObject
  • Find value with reflection

Not sure if these will be any faster.

DocumentCollection could have latest id value stored in a variable. Should be lazy so won't be calculated unless needed. This should improve performance.

ttu avatar Jun 22 '18 07:06 ttu

Interesting idea to store the latest ID in variable. I will try to use this approach.

qpooqp avatar Jun 23 '18 16:06 qpooqp

Hello. I just want to give you some update.

I'm quite far, but still struggling with some cases. Main problem is when there is string ID. I can't easily compare it and when it gets mixed up with int IDs then it causes problems too. But i will try to resolve this and then i will let you know.

qpooqp avatar Jun 28 '18 16:06 qpooqp