Query profiles

[novoj]

In current API there are some parts that complicate the storefront implementation. Namely:

• set of "persistent filters" - such as PriceValidIn (now), attributeEquals (status = ACTIVE)
• HierarchyExcluding inserting set of categories with having attributeEquals (visibility = INVISIBLE)
• QueryPriceMode set always on WITH_VAT
• FacetGroupsConjunction, FacetGroupsDisjunction, FacetGroupsNegation that is always derived from referenced entitites having attributeEquals (mode = NEGATIVE) and so on

These "relations" needs to be wired up in the storefront middleware and although doable they complicate the implementation, require caching (and proper invalidation) and may introduce subtle bugs if some of these "mandatory" constraints is omitted from the query by mistake.

That's why the idea of "query profiles" looks promising on the first sight. Let's say we can create a named profile using the evitaDB API that would allow to define declarative "rules". Then the client would just apply the profile for the query in following way:

query(
	collection(Entities.PRODUCT),
	filterBy(
		and(
			entityPrimaryKeyInSet(primaryKey),
			entityLocaleEquals(locale)
		)
	),
	require(
		profile("b2c")
	)
)

When such profile is used, server side (evita query engine) would automatically enrich the query of a new conjunctive filters / additional requirements that were associated with this profile. This would allow shifting a lot more logic to the server part that maintains the data set and make the logic on frontend / middle-ware a lot easier and maintainable. Also moving the selection logic to standard constraints and formula calculation tree would allow us to reuse existing core cache / invalidation baked in evitaDB. This removes the necessity to handle cache in the middle-ware at all and promises the correct and immediate invalidation process in case the data is changed.

The profiles should be able to:

• define filtering constraint for one or multiple entity types
• define requirement constraint for one or multiple entity types

Of course we'd need to extend certain constraints to be able to accept a sub constraint filter instead of exact primary keys (HierarchyExcluding, FacetGroupsConjunction, FacetGroupsDisjunction, FacetGroupsNegation) - which doesn't directly relate to this issue of query profiles, but can be solved separatedly.

The query profiles should be composable - so that we could use multiple profiles in a single query profile("b2c","visible").

Another thing that comes to my mind in connection with query profiles is implications to security. The profiles might play nicely with something like EdgeDB access policies or table permissions in SurrealDB - so when we introduce "logging in" process to the Evita session establishing proces we could enforce using one or more profiles for entire session based on the logged in user account credentials and thus limit the access of the user to the certain data.

We should also extend queryTelemetry hint to allow returning complete query composed from different profiles to allow debugging problems by the developers (because part of the query composition is hidden for them on the server side).

[novoj]

Recording of original private discussion:

@lukashornych: I think one way to declare rules for adding constraint to containers could be using relative/absolute paths. Client would specify path of parent container and specify either add or remove operation and child constraint to add/remove.

For example:

{
  location: "hierarchyWithin",
  operation: "ADD",
  childConstraint: "excluding(attributeEquals('visibility', INVISIBLE))"
}

which would find all hierarchyWithin constraints and add new children to them. We could then use our query parser for actual constraints parsing.

One think that I don't know if would acceptable is that in case of GQL, syntax of constraints in rules would be different. Also, we maybe probably need to implement some sort of wildcards for query parser for things like classifiers to be able to specify exactly which constraints to select.

@novoj: Interesting idea, this resembles me an existing approach in io.evitadb.api.query.QueryUtils#findConstraint(io.evitadb.api.query.Constraint<?>, java.lang.Class<T>). But in Java we could afford programmable predicate logic, that is much more versatile than declarative one.

@lukashornych: Exactly, but I think for mentioned rules on client side this simplified declarative approach could be sufficient. We could add some more logic in to the path field such as hierarchyWithing('category',*,*) which would use only those with category collection.

This extension of path maybe sufficient even for remove operations.