Adding GitHub search to Meili docs

[erlend-sh]

In similar fashion to this multi-site search, docs.meilisearch.com could include GitHub Issues (and eventually Discussions) as an additional search source:

Originally posted by @erlend-sh in https://github.com/meilisearch/MeiliSearch/discussions/629

[tpayet]

In order to achieve GitHub search to Meili docs we need 2 things:

• GitHub issues and pull request indexer
• modify docs-searchbar.js in order to display multi-index result.

The first one can be implemented asap but I would like some feedback from @curquiza & @qdequele on how to modify docs-searchbar. How complicated would that be?

[curquiza]

About docs-searchbar.js, it's doable, but it would be waaay easier if MeiliSearch would handle the multi-indexes search instead of adapting our tools...

The search in the library is done here.

In any case, we have to change the indexUid field: it could be an array or a string (currently just a string is allowed) and we could adapt the search depending of this type. With this solution, the new version would be non-breakable. But again, it would be great if MeiliSearch would handle multi-indexes search 😬

(I ping you @Kerollmops because it might be interesting for you)

After talking with Thomas, I realized I didn't understand well the behavior of the multi-indexes search! I thought that MeiliSearch would search in many indexes, would get many groups of results, and would do a final sort among these groups. Which is not what the multi-indexes search would be.

So, here are the steps to add the GitHub information to the documentation search bar (the order matters!!):

1. scrape our documentation website and put them into an index (let's call it docs). This step is already done in our GHA.
2. get the GitHub data with the GitHub API or by scraping.
3. push the data into the same index (docs). The data must have the same format (= same fields) than the data already present in the docs index so that we can display them the same way. The title (with the green background) could be GitHub.

That's not exactly the display that @erlend-sh suggested but that would be the easiest integration to do. Especially because It would not need to change the docs-searchbar.js library 🙂

[Kerollmops]

Hum, ok yeah.

The multi-indexes search doesn't mixup the results of the different indexes you are searching in, it only returns an array with the different results you would have found if you had requested each indexes independently. The engine can't fix them up because the schema is different and therefore it don't know which fields are better than another.

Your new search bar setup seems good, but are you sure you can fit the Github documents into the documentations index? Is the schema equivalent? Will it work?

[curquiza]

The multi-indexes search doesn't mixup the results of the different indexes you are searching in, it only returns an array with the different results you would have found if you had requested each indexes independently. The engine can't fix them up because the schema is different and therefore it don't know which fields are better than another.

Ok so with my suggestion I indeed don't need the multi-indexes search.

Your new serach bar setup seems good, but are you sure you can fit the Github documents into the documentations index? Is the schema equivalent? Will it work?

That might be complicated but possible. I need to test that and let you know 🙂 I think that could work. The question is more about the apparition priority of the GitHub results in the search bar. I need to find the right settings for that.

[curquiza]

Here is an issue I've just opened for a user. It's exactly our kind of use-case. The answer I wrote down clarifies a lot about what to do.

Here is the hierarchy we thought with @tpayet:

• lvl0 -> GitHub
• lvl1 -> Issue or Pull Request
• lv2 -> title of the issue/PR
• content -> the comments of the issue/PR

[guimachiavelli]

We have discussed this internally and are against adding search issues to the main searchbar. We are afraid it will create too much noise—searching for something and ending up on an ongoing discussion about something that has not been solved might be frustrating and confusing, especially for more inexperienced users. Users come to the docs for unambiguous answers and usage instructions, something which issues only occasionally supply.

@dichotommy does believe having a separate search bar might be useful for users who have been unable to find the answer to a problem.