attributesToSnippet not working as expected

[jakeprice-me]

Description

I'm trying to display the first 40 tokens/words from the content of a document by default (before any search) as opposed to the entire content.

I've seen previous issues that have recommended using highlight_affix_num_tokens and I am indeed using this but this is activated after a search, so it's not what I'm after in this case.

Steps to reproduce

Using instant-meilisearch I can accomplish what I'm trying to do with the below code using Snippet, but it doesn't appear to work with Typesense.

  instantsearch.widgets.configure({
      attributesToSnippet: ['content:40'],
  }),
  instantsearch.widgets.infiniteHits({
    container: '#results',
    templates: {
      item: `
            <div id="title-snippet">
                <div id="title">
                    <h1><a href="{{ url }}" target="_blank">{{ title }}</a></h1>
                </div>
                <div id="link">
                    <a href="{{ link }}" target="_blank">{{ link }}</a>
                </div>
                <div id="snippet">
                    <p>{{#helpers.snippet}}{ "attribute": "content" }{{/helpers.snippet}}</p>
                </div>
            </div>
            <div id="metadata">
                <date>{{ date }}</date>
                <div id="type">
                    <span class="label_{{ types }}">
                        <a href="https://example.com/{{ types }}" target="_blank">
                            <i class="las la-folder"></i>
                            {{ types }}
                        </a>
                    </span>
                </div>
            </div>
      `,

Expected Behavior

Show first 40 tokens/words of document's content (with the ellipses ... at the end).

Actual Behavior

Shows entire document's content.

Metadata

Typesense Version: 0.22.2 OS: Linux (Fedora 36) instantsearch.js Version: 4.43.0 typesense-instantsearch-adapter: 2.4.1-3

Do let me know if you need anything further to help debug. Any help is appreciated!

[jasonbosco]

@jakeprice-dev Could you put together a minimal example in codesandbox showing the issue? I can help debug from there

[jakeprice-me]

@jasonbosco thanks for coming back on this!

I should have clarified that I'm not a JS programmer and my exposure to JS is limited. I'm using Typesense within my private network and not exposing it to the net. With that context I'm aware that it may be something I'm doing wrong - is this not an issue you've seen with the attributesToSnippet option before? I do want to avoid wasting your time if the option works as expected for you? I'm certainly not asking for debugging help to find a bug in my code!

Otherwise, sorry it's taken a little while for me to reply. I've given quite a go at trying to get an example running in code sandbox, but really not getting anywhere at all. I eventually went down the road of spinning up Typesense on a cheap VM and seemed to be getting somewhere with code sandbox, but then just kept running into CORS issues.

It's not the end of the world, but certainly interested to know if this is an issue for me only, as that would then help me identify a bug in my own code. Thanks again.

[jasonbosco]

@jakeprice-dev I essentially didn't understand what you meant by "before any search". So I figured it would be easier if you had a demo I could play around with and then if there's an issue in the typesense-instantsearch-adapter I can debug the adapter with your demo, and make sure any potential fixes work well.

In any case, could you clarify what you mean by "before any search"? Did you mean a query with q=*? ~If so, highlight_affix_num_tokens will still work~, but you need to specify it in additionalSearchParameters when instantiating the adapter and not in the configure widget.

[jakeprice-me]

Sorry @jasonbosco so what I mean by "before any search" is that by default when I load my search interface, and before I have ran a query I want to display a snippet of the content (first 40 words/tokens etc) as opposed to the whole content of a document. I can get that behaviour with instant-meilisearch.

"Before any search" does indeed seem to be the same as if I query with q=*. I've removed attributesToSnippet from the configure widget, but highlight_affix_num_tokens doesn't seem to have any effect on q=*. It works if I query something like q=test however.

Hopefully the below images should help clarify?

q=* / before any search

You can see that the whole content of a document is returned as opposed to the first 40 words/tokens for example.

q=Brocklehurst

Snippets the finding as expected.

Here's my code (with the api key removed). More than happy to put my interface on a VM with some dummy data so you can see the behaviour in action if you need me to?

const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
  server: {
    apiKey: '', // Be sure to use an API key that only allows searches, in production
    nodes: [
      {
        host: 'ts.int.ppn.sh',
        protocol: 'https',
      },
    ],
  },
  // The following parameters are directly passed to Typesense's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  queryBy is required.
  //  filterBy is managed and overridden by InstantSearch.js. To set it, you want to use one of the filter widgets like refinementList or use the `configure` widget.
  additionalSearchParameters: {
    query_by: 'title,content',
    highlight_affix_num_tokens: 20,
    hidden_hits: "archive",
  },
});
const searchClient = typesenseInstantsearchAdapter.searchClient;

const search = instantsearch({
  searchClient,
  indexName: 'log',
  routing: true,
});

search.addWidgets([
  instantsearch.widgets.searchBox({
    container: '#searchbox',
    placeholder: "Search...",
    autofocus: true,
    searchAsYouType: true,
    showReset: true,
    showSubmit: false,
  }),
  instantsearch.widgets.configure({
    hitsPerPage:50,
  }),
  instantsearch.widgets.refinementList({
    container: '#refinement-list',
    attribute: 'types',
    limit: 100,
    sortBy: ['name:asc'],
    templates: {
      item: `
            <span class="label_{{ label }}">
                <a href="https://log.int.ppn.sh/types/{{ types }}" target="_blank">
                <a href="{{url}}" style="{{#isRefined}}font-weight: bold{{/isRefined}}">
                    <i class="las la-folder"></i>
                    {{ label }}
                </a>
            </span>
      `,
  }}),
  instantsearch.widgets.sortBy({
    container: "#sort",
    items: [
      { label: "Default", value: "log" },
      { label: "Date (Oldest First)", value: "log/sort/date:asc" },
      { label: "Date (Newest First)", value: "log/sort/date:desc" },
    ],
  }),
  instantsearch.widgets.stats({
    container: '#stats',
  }),
  instantsearch.widgets.infiniteHits({
    container: '#results',
    templates: {
      item: `
            <div id="title-snippet">
                <div id="title">
                    <h1><a href="{{ url }}" target="_blank">{{#helpers.snippet}}{ "attribute": "title" }{{/helpers.snippet}}</a></h1>
                </div>
                <div id="link">
                    <a href="{{ link }}" target="_blank">{{ link }}</a>
                </div>
                <div id="snippet">
                    <p>{{#helpers.snippet}}{ "attribute": "content" }{{/helpers.snippet}}</p>
                </div>
            </div>
            <div id="metadata">
                <date>{{ pretty_date }}</date>
                <div id="type">
                    <span class="label_{{ types }}">
                        <a href="https://log.int.ppn.sh/types/{{ types }}" target="_blank">
                            <i class="las la-folder"></i>
                            {{ types }}
                        </a>
                    </span>
                </div>
            </div>
      `,
    empty: `
        <div id="no-results">
            <p>No results for <span id="no-results-result">{{ query }}</span></p>
        </div>
    `,
    },
  }),
]);

search.start();

[jasonbosco]

@jakeprice-dev Got it. I misspoke earlier. It turns out that highlight_affix_num_tokens only kicks-in when q is set to something other than q=*. We need to add support in Typesense to allow snippeting for q=* queries. Could you open a separate issue in https://github.com/typesense/typesense/issues for this?

In the meantime, one thing you could do is use transformItems in the infiniteHits widget and snippet on the client-side.

[jakeprice-me]

That's great thanks @jasonbosco, have it on my todo list to raise an issue over on the typesense repo, so will do it asap :)