LintDB
LintDB copied to clipboard
A user should be able to filter documents
As a user of LintDB, I want to be able to filter by metadata in a search so that I can more accurately retrieve my data.
Acceptance Criteria
- We can filter on a. strings b. dates
- Metadata can be returned in the search response.
We need to consider how data will be stored on disk.
- Flatbuffers needs to precompile the data structures, which prevents users from specifying a schema. Should a user specify a schema to store metadata?
We also need to consider how we'll filter data without duplicated the metadata.
- My first thought is that we need to create inverted lists for the attributes and create a retrieval process that looks like document-at-a-time. We might want a query engine to accomplish this.
Notes
I'm curious who has a good version of a query engine to think about. Quickwit is awesome and might have something similar?