Add index

[n-peugnet]

trafficstars

Hi @jamesmoss i'm so glad you merged my previous PR so fast. I started to implement indexes. but I still don't feel like the job is finished. I make this PR as a draft so you can give me your opinion.

I only did HashIndexes yet but I plan to make some more. I'm thinking for example about a FulltextIndex that could index words in strings.

[n-peugnet]

I still have one problem with this approach:

Before, it was possible to edit the values of the document from an external source and still have coherent results. This is not possible anymore when the query is using the index. I'm wondering how big of a problem it is. I guess if one wants to be able to do it, one could just don't add any index. Maybe I could also add a function to regenerate the indexes to make sure they are in sync with the documents.

~~Or I could also test the diff between the index file date and the repository's directory date.~~ edit: This is not going to work in every situation

thing left to do (maybe)

• [ ] ~~compare dates to regenerate index~~
• [ ] cache index (I am not sure how to do this) or add a cached bloom filter

[jamesmoss]

Before, it was possible to edit the values of the document from an external source and still have coherent results.

For me this was a major design decision and a benefit of Flywheel. I know indexes are opt-in right now which is good but I'd been keen to not lose that and still get the performance improvements.

Can you think of another approach we could take? it might be impossible and I might be asking too much here.