Attempting to import a large index is crashing on mobile Safari

[mgburns]

I'm working on a Vue/Gridsome project that utilizes this library for search. We have some very lengthy docs that were causing client-side indexing to take a while (and blocking search). We are attempting to pivot and generate the index at build time and serialize to disk, and then import in the browser on initial page load.

Everything is working great on desktop and Android, but I noticed that mobile iOS browsers are crashing. We first noticed it with a 23 MB index (source) but it still occurred after pruning it down to 10 MB. We've isolated the issue to a flexsearch.import() call. It loads the file and starts the import, but never completes. After ~10 seconds the page jumps a bit, and then crashes with A problem repeatedly occurred on "<site url>".

Relevant Flexsearch configuration on the server:

{
  tokenize: "forward",
  // This is essential the Unicode-friendly version of the default split (/W+/)
  // @see https://www.regular-expressions.info/unicode.html
  split: /[\p{Z}\p{S}\p{P}\p{C}]+/u,
  doc: {
    id: "id",
    field: ["prettyTitle", "htmlContent"],
  },
};

And on the client:

{
  tokenize: "forward",
  // This is essential the Unicode-friendly version of the default split (/W+/)
  // @see https://www.regular-expressions.info/unicode.html
  split: /[\p{Z}\p{S}\p{P}\p{C}]+/u,
  doc: {
    id: "id",
    field: ["prettyTitle", "htmlContent"],
  },
  async: true,
};

I also tried setting worker: 4 to offload the import to a web worker but that didn't appear to have any impact.

My hunch is that I'm running up against some sort of mobile Safari resource limit (either max execution time for JS or memory exhaustion). I'm testing on an iPhone 12 Mini (256GB) running iOS 14.4.2 Flexsearch version is 0.6.32.

[ts-thomas]

Thanks a lot for this hint: /[\p{Z}\p{S}\p{P}\p{C}]+/u. Could you give me a short example please, where this notation gets an advantage over /W+/?

I can recommend you to use the new version 0.7.x which has improvements for memory consumption. Here is a good configuration to get a small memory footprint: https://github.com/nextapps-de/flexsearch#memory-allocation

• use "strict" tokenizer
• use a small "resolution" value
• use encoder "latin/extra.js"
• do not use context index
• use a "minlength"
• do not use a store when your application holds a reference to your data, otherwise provide a store definition to just keep data you need in your results

[mgburns]

Thanks for the response! We'll give 0.7.x a shot and see if it resolves the crashes on mobile Safari.

Thanks a lot for this hint: /[\p{Z}\p{S}\p{P}\p{C}]+/u. Could you give me a short example please, where this notation gets an advantage over /W+/?

We had to switch to this pattern to support Hindi -- the following produced no results with /W+/: मुकेश