Case insensitive filtering and sorting

[mercmobily]

I realise there is a discussion going on in terms of expanding the querying abilities in dstore #34 . I have added my humble 2c to that discussion.

However, I am having a practical issue now where I cannot even really complete the test cases for my store tutorial as my DB layer is case insensitive in matching and sorting, whereas Memory does care about case. This gave me some interesting head-scratching moments where a newly added element in a cache collection ended up in a seemingly random spot in the dgrid (because the queryEngine was placing it considering case sensitiveness). While I think it's great that a direction in terms of querying is being taken, I was hoping you'd consider a pull request to make queries and sorting case-sensitive now (according to the store's corresponding attributes).

I realise that there is an ongoing discussion in #34 in terms of query abstraction layer. But, at the moment I am not even able to complete a very simple use case, as my DB server is case-insensitive in filtering and sorting, whereas Memory.js does care about case. This created an interesting head-scratching moment when queryEngine places items in seemingly random spots.

I meant to create a PR with this, but Github kept on saying "there is nothing to compare". However, the changed file is:

https://github.com/mercmobily/dstore/blob/caseinsensitive/objectQueryEngine.js

The list of changes:

https://github.com/mercmobily/dstore/pull/1

It has very small changes, aimed at making filtering and sorting case-insensitive based on store's configuration.

Thanks as ever.

[mercmobily]

Kris wrote on my own fork:

Adding better support for case insensitivity sounds good. The concern I have about this particular approach, is that I am hesitant to make queries be more context-sensitive. Generally, I think a query should fully and explicitly specify what matches and doesn't match, and with this approach a query behaves differently based on store configuration. And this also would make it difficult to have queries with mixed behavior (some properties case sensitive, some note).

For sorting, I think it would be quite reasonable to simply add another property to the sort objects that would specify case sensitivity.

For filtering, I would like to make the query building capable of handling this. Currently the Memory store should support {property: /^aNyCaSe$/i}. Of course, we have incomplete support for this, since presumably the Request may serialize this poorly for the query parameter.

[mercmobily]

Case sensitive sorting: having a property to the sort object would be fine. But we then need to pass that to the server somehow as well.

Case sensitive filtering: having a query building mechanism is interesting, but complex. At the moment, we have such an example of an uni-directional solution: Memory supports regexp, but they aren't really translated into something meaningful when they are asked to the server.

You say that case-sensitiveness it should be query-based; I say that it should be store-based; what if I say "let's make it field-based"? Here is what I am thinking.

1. I create an objectQueryEngine factory. In the factory, you specify what fields are searchable, and how they are searchable. For example, you can specify that "nameStartsWith" is a searchable field, and that it will search for objects where the property 'name' starts with XXX. Or, you could define a property called "nameOrSurnameContain" (which does what I mean here).

2. When creating a store, you create the appropriate objectQueryEngine so that specific fields mean specific things

3. On the server side, you make sure that those "made up" fields have the same filtering meaning as they do on the client

This way, you can still keep the simple key/value nature of a query using a store (which also makes it easier for consumers to use) while still have the power of making more complex queries.

I realise this is now a "query language" issue, but it is all related, really.

[mercmobily]

Kris, I am exploring this right now. It's a little easier for me as I already have a server that accepts "augmented" filtering options. So, testing is easier. I was just wondering whether you were open to the idea of "super-fields", or if you thought that having the ability of making up complex queries really is the way to go.

[kriszyp]

I am assuming that "super-fields" could be implemented by overriding _renderFilterParams to generate the specific parameters for a given server.

[schallm]

I would like to add +1 for case-insensitive sorting. I'm working on upgrading to dgrid 0.4 right now and was hoping dstore had an answer for this. A property for this does seem like the correct approach. Any ETA on this decision?

Really enjoying dstore.

[jgrayson-apl]

+1