Returning 0 in facetsDistribution (MeiliSearch 0.21 and later)

[bidoubiwa]

New facetsDistribution Behavior in v0.21

In v0.21 facetsDistribution will stop returning facets that contains 0 documents (see meilisearch/transplant/issues/234). Contrary to v0.20

example: v0.20

"facetsDistribution": {
        "genre": {
            "adventure": 1,
            "fantasy": 1,
            "romance": 0,
            "Science fiction": 0,
            "thriller": 0
             // ...
        }
    },

v0.21

"facetsDistribution": {
        "genre": {
            "adventure": 1,
            "fantasy": 1
        }
    },

Problem

Because some uses cases depend on having the 0 returned, it might be breaking.

Example: search params:

{
   filter : ["title=Hamlet",["genre=comedy", "genre=fiction"]],
   facetsDistribution: ["title", "genre"]
}

might return

"facetsDistribution": {
        "genre": {
            "fiction": 1,
        },
       "title": {
          "Hamlet": 1
       }
    },

Since comedy: 0 is not present in the facetsDistribution you might have a hard time rendering the UI.

Solution

All fields

There are two solutions to this problem, either you cache all fields and upon search return, and based on the missing fields in facetsDistribution you append facetsDistribution with all the missing fields and a 0 value.

Independent of the filters it returns all facet fields:

"facetsDistribution": {
        "genre": {
            "adventure": 1,
            "fantasy": 1,
            "romance": 0,
            "Science fiction": 0,
            "thriller": 0
             // ...
        }
    },

Only filtered fields

Or we only add the missing fields that are also present in the settings.

On a filter with genre=comedy and genre=fiction:

{
  facetsDistribution: { genre: { fiction: 1, comedy: 0 } }
}

Solution 1: Only return filtered fields

Step 1: cache facets

On arrival, facets will look like this

["title=Hamlet",["genre=comedy", "genre=fiction"]]

The objective is to retrieve all facets in their respective category. The code is accessible in this TypeScript Playground or in this javascript playground

this will output the following:

{
    "genres": [
        "drama",
        "comedy"
    ],
    "title": [
        "hamlet"
    ]
}

Step 2: Add missing 0 fields in fieldsDistribution

Following the previous example, imagine upon a search on MeiliSearch you receive the following answer:

"facetsDistribution": {
        "genre": {
            "fiction": 1,
        },
       "title": {
          "Hamlet": 1
       }
    },

comedy: 0 is missing in genre.

Using our cache we can add it back:

 const addMissingFacetZeroFields = (
   cache,
   distribution
 ) => {
   if (cache && distribution) {
     for (const cachedFacet in cache) {
       for (const cachedField in cache[cachedFacet]) {
         // safety check: ensure facet key exists
         if (!distribution[cachedFacet]) continue;

         // if cached field is not present in the returned distribution
         if (!Object.keys(distribution[cachedFacet]).includes(cachedField)) {
           // add 0 value
           distribution[cachedFacet][cachedField] = 0
         }
       }
     }
   }
   return distribution
 }

After using this function, the facetsDistribution looks like this:

"facetsDistribution": {
        "genre": {
            "fiction": 1,
            "comedy": 0
        },
       "title": {
          "Hamlet": 1
       }
    },

Keeping all the 0 facets distribution

First step is to make a placeholder search. This will contain all the possible facets values. Then we need to retrieved it:

function getFacetsFromDefaultDistribution(
  facetsDistribution: FacetsDistribution
): FacetsCache {
  return Object.keys(facetsDistribution).reduce((cache: any, facet) => {
    const facetValues = Object.keys(facetsDistribution[facet])
    return {
      ...cache,
      [facet]: facetValues,
    }
  }, {})
}

This will return an object with the following structure:

{
   "genres": ["comedy", "fiction"],
   "title": ["Hamlet"]
}

With this information we will be able to auto-complete the missing facet values in the returned facetDistribution upon search the same way we did in the first section of this issue.

⚠️ Important: this solution works only for filter using only the array syntax.

Example:

Works ✅:

const filter = ["title=Hamlet",["genre=comedy", "genre=fiction"]]

Works ✅:

const filter = ["title=hamlet"]

Incorrect ❌, does not work:

const filter = "title=Hamlet AND (genre=comedy OR genre=fiction)

Incorrect ❌, does not work:

const filter = "title=hamlet"

Incorrect ❌, does not work:

const filter = "title=Hamlet AND (genre=comedy OR genre=fiction)

[tao]

With Solution 1: Step 2, I think you'll need to update this to include the following line:

// remove filter if non existant
if (!distribution[cachedFacet]) continue;

This is because you can run into an error if you switch indexes dynamically and the filter doesn't exist in the new index:

let cache = { locale: { en: 219 }, type: { interviews: 219 } };
let distribution = { locale: { en: 9 } };

// will throw error because `type` doesn't exist in distribution
if (!Object.keys(distribution[cachedFacet]).includes(cachedField)) {
    ...
}

So the code could be updated to ensure the key exists before checking if the cachedField exists in the facet.

 const addMissingFacetZeroFields = (
   cache,
   distribution
 ) => {
   if (cache && distribution) {
     for (const cachedFacet in cache) {
       for (const cachedField in cache[cachedFacet]) {
         // safety check: ensure facet key exists
         if (!distribution[cachedFacet]) continue;

         // if cached field is not present in the returned distribution
         if (!Object.keys(distribution[cachedFacet]).includes(cachedField)) {
           // add 0 value
           distribution[cachedFacet][cachedField] = 0
         }
       }
     }
   }
   return distribution
 }

[bidoubiwa]

Indeed I forgot to think about this use case. I'm updating the step 🔥

[ar-siddiqui]

Are we hoping to go back to returning 0 values in the future versions? We do not like meilisearch not returning 0 values.

For our case for example we have an attribute file_type that can be either document, pdf, movie, or image. There is no easy way to configure the frontend client to show movies still if someone has already selected an image.

[bidoubiwa]

You can ask about that in the product repository where the design of Meilisearch is decided. You can open a discussion with your request :)

[bidoubiwa]

Not critical, closing