Bootstrap-3-Typeahead icon indicating copy to clipboard operation
Bootstrap-3-Typeahead copied to clipboard

Bloodhound Tokenizer Changes Don't Change Matching

Open ddn opened this issue 7 years ago • 0 comments

I'll preface by saying I'm sure there is a mistake I am making here somewhere, but I can't figure it out.

I've specified nonword tokenizers, but used a callback so I can view the tokenization.

    datumTokenizer: function(d) { 
      console.log(Bloodhound.tokenizers.nonword(d));
      return Bloodhound.tokenizers.nonword(d); 
    },
    queryTokenizer: function(d) { 
      console.log(Bloodhound.tokenizers.nonword(d));
      return Bloodhound.tokenizers.nonword(d); 
    }

I can see the local data source tokenized correctly, i.e.:

Value: Barbell Roll-Out is tokenized as (3) ["Barbell", "Roll", "Out"]0: "Barbell"1: "Roll"2: "Out"

Same correct tokenization with a search of "Roll Out"

(2) ["roll", "out"]

Nonetheless, it won't match unless I type the hypen that is in the original value.

Is there some missing flag I should be using here?

(For completeness, if I change the tokenizer to whitespace, the word roll-out is tokenized as one word, as it should be splitting on whitespace.)

ddn avatar Jan 16 '18 21:01 ddn