Stein
Stein copied to clipboard
Implemented more complex seach capabilities using Mango (MongoDB like) query syntax
Issue #16 asks for more complex queries.
The solution
Using the solution in this PR, complex queries is written as a PouchDB selector, which is the same as CouchDB selector, which is a Mango query language, which is inspired by MongoDB query language.
Query parsing and rows matching are performed by using pouchdb-selector-core. All comparison operators ($eq, $ne, $gt, $lt, etc.) and logical operators ($and, $or, etc.) have been implemented by pouchdb-selector-core.
The reasonings
Reasonings of using MongoDB query language and pouchdb-selector-core:
- Current way to write search query still works. Performing search with
?search={column1:value1,column2:value2}
still works, and it is the same as?search={column1:{$eq:value1},column2:{$eq:value2}}
. It means it retains backward compatibility. - MongoDB is popular, thus it's query language is familiar. In my opinion, for Stein case, it is better to write the query like this than in other query language like SQL or defining our own custom query format.
- pouchdb-selector-core is used internally by PouchDB and it is well maintained. PouchDB is a mature library and I have not yet met any problems using PouchDB selector. There are another MongoDB query matcher called mongo-parse, but it has security problem (script eval injection).
The implementation
I don't use existing match function in pouchdb-selector-core, which is matchesSelector
, because it only perform match for one row. I think it is not efficient to process the query every row. The query doesn't change so it is better to only process the query once and then use it to filter the dataset. This is why I use massageSelector
and filterInMemoryFields
in the implementation because these two functions is performed matchesSelector
.
One thing to note for users, query values should always be string or null since data from Stein are always string or null even though it is a number. For example, ?search={"year":{"$eq":2020}}
would not work because the value (2020) should be a string ?search={"year":{"$eq":"2020"}}
.
Performing to string conversion for all field values is possible but difficult because using logical operators the query object can be deep.