Viewport rules

[novoj]

Viewport rules

Often the product listing page contains listings in differently sorted forms. The typical example is the "Top Sellers" block, which lists the exact same products as the main listing on the page, but sorts them differently. If we could merge these two listings into a single query that returns two results, we could benefit in two ways:

1. we wouldn't be forced to declare the same complex set of filtering constraints and requirements in two different queries.
2. the database engine could reuse the computed result of filtered products to satisfy both queries at once - just applying different sorting and paging requirements.

So we end up with a faster and simpler system for this use case, which is ultimately what we want.

The idea is to optionally define segment rules for paging requirement that could integrate different slices of the result with completely different sorting. Let's have an example scenario where we want to have:

1. first 3 positions to be occupied by top-selling products
2. next 2 positions to be new products in category (randomly selected)
3. next 5 positions to be currently promoted products (randomly selected)
4. then the rest of the products to be sorted by artificial priority set in the backend ERP system

{
  queryProduct(
   orderBy: {    
     segments: [
        {
          orderBy: {
            attributeOrderedQuantityNatural: DESC
          }
          limit: 3
        },
        {
          onEntitiesMatching: {
            attributeNewEquals: true
          }
          orderBy: {
            random: true
          }
          limit: 2
        },
        {
          onEntitiesMatching: {
            attributePromotedEquals: true
          }
          orderBy: {
            random: true
          }
          limit: 5
        },
        {
          orderBy: {
            attributeOrderNatural: DESC
          }
        }
      ]   
   }
  ) {
    recordPage(
      number: 1
      size: 20
    ) {      
      data {        
        primaryKey
        attributes {
          code
        }
      }
    }
  }
}

The last ordering has no limit specified so that it can be used for the rest of the entities/products not used in the previous segments. If the limit is defined, the rest of the matched products will be appended in natural ASC order of their entity primary key.

The initial proposal aims to handle segments in sequential order - focusing primarily on the first page of the listing. It can be extended to define rules for arbitrary pages (e.g. every odd page, starting with page 3), etc. This advanced scenario is only worth pursuing if the basic one pays off in practice.

Spacing rules

Another common scenario is to skip certain positions in the listing to display non-product data such as advertising or content blocks. While this feature can be implemented using the existing [strip] (https://evitadb.io/documentation/query/requirements/paging?lang=graphql#strip-recordstrip) feature, it requires careful calculations on the application side. If we could integrate spacing rules into the viewport composition, it could be reliably calculated on the database side, and the client application could simply use the existing page requirements, which are much easier to use.

We could build on the foundation of the previous suggestion and just add a spacing rule. Let's have an example where we want to have:

1. one ad block on each page, up to page 6
2. an additional block of blog post teasers on the first three even pages

{
  queryProduct {
    recordPage(
      number: 1
      size: 20
      spacing: [
          {        
            gap: {
              size: 2            
              onPage: "($pageNumber - 1) % 2 == 0"  
            }
          },
          {        
            gap: {
              size: 1
              onPage: "$pageNumber % 2 == 0"
            }
          }
        ] 
    ) {      
      data {        
        primaryKey
        attributes {
          code
        }
      }
    }
  }
}

Note: only first matching rule will be applied. Implementation note: we will implement our own ANTLR grammar for calculations (we could start with something like this example)

Sticking sorting randomness

Another scenario that the client solves is the reliability of the output to the end user. Consider this situation - we have 10 products we want to promote on the first 3 positions of the listing. In order to cycle them all, we choose to sort them randomly (so every time there are different 3 promoted products out of 10 available). But this could be confusing for the user who refreshes the page or lists through additional pages. This situation can be easily solved by passing the seed from the client side to the random function, which could generate the seed per user and store it in their session cache.

Example usage:

{
  queryProduct(
    orderBy: [
      {
        randomWithSeed: 12348
      }
    ]
  ) {
    recordPage {
      data {
        primaryKey
        attributes {
          code
        }
      }
    }
  }
}

[novoj]

I've discussed this matter also with o1-preview and refined naming a little bit:

• https://chatgpt.com/share/66f155c8-b5b0-800a-8f9f-afe166f62a0a

Finally, I tried to let it explain what logic the final query proposal represents with empty LLM context and it was picked exactly as we intended: https://chatgpt.com/share/66f15631-47e4-800a-85fd-0ba070cdff59

Let's hope the humans will comprehend it the similar way :)

[lukashornych]

There can be potential breaking change in generated GraphQL and REST APIs if any reference schema contains reference to external entity (not managed by evitaDB) either as referenced entity type or reference group type. In that case, generated OrderContainer* types for these external entity references, will have different suffix in their name. Content-wise, it should not be breaking.

For example, in our demo dataset, there are referenced tagCategory and stockGroup there are external to evitaDB and those will use new OrderContainer* type names in queries.

[novoj]

Issue is almost done. We wait only for @lukashornych to finalize REST / GraphQL support for segments and we can also finalize the documentation.