tw-full-text-search
tw-full-text-search copied to clipboard
Pipeline tiddlers
Lunr.js allows you to hook into the pipeline it uses to massage documents in order to index them; currently I have a single custom pipeline function that expands synonymns in queries.
It might be helpful to expose the pipeline to external clients, such as other plugins - this way we can prototype new ideas outside of the plugin itself, and it makes the plugin more flexible.
I'm thinking the plugin could load modules with module-type: lunrpipelinefunction
, and a configuration list variable could specify the ordering of the pipeline functions. A few questions about the configuration, though:
- How do we make sure the pipeline includes the default functions if it's a freeform list? Do we need to make such a guarantee?
- You might not want to - think of replacing the default stopwords/stemmer with non-English variants!
- How do we keep well-meaning but non-advanced users from hosing their wiki by throwing in too many pipeline plugins? Even for knowledgable users, arranging the functions in the pipeline is somewhat of an art!
A preview pane of how the pipeline would massage data would be very cool, but probably a bit beyond the scope of this issue.