Selectize with 'dynamic' choices: front-end search hides back-end results

[Actionb]

For model choice fields with a selectize widget, the results of the back-end query are filtered again in the front-end (by sifter, I assume). This hides results the back-end query has found.

Take the State model from the test app:

class State(models.Model):
    code = models.CharField(
        verbose_name="Code",
        max_length=2,
    )

    name = models.CharField(
        verbose_name="Name",
        max_length=20,
        db_index=True,
    )

    class Meta:
        ordering = ['name']

    def __str__(self):
        return self.name

Assume you have a widget like this:
Selectize(search_lookup=['name__icontains', 'code__icontains'])
And a State like this:
State(name='foo', code='bar')

In the form, you want to find the state by its code, bar. The back-end query will find that state no problem, and the view will add it to the available options "options": [{"id": 1, "label": "Foo"}]}. But then the front-end will do another search on these options for the search term bar - and will not find any matches.

[jrief]

I just tested here: https://django-formset.fly.dev/bootstrap/state

If I type "son", many counties ending with …son are found. If I type "ky" many counties in Kentucky are found. Yes, it's true that tom-select applies its own filter, but so at least it highlights the matching string.

Do you see any possibility in removing stifer.js, while initializing tom-select?

[Actionb]

I just tested here: https://django-formset.fly.dev/bootstrap/state

If I type "son", many counties ending with …son are found. If I type "ky" many counties in Kentucky are found. Yes, it's true that tom-select applies its own filter, but so at least it highlights the matching string.

When both the back-end and the front-end do the same filtering, then there isn't an issue. The issue arises when the front-end filters differently from the back-end. In the example I gave, the back-end queries two fields (name and code) and returns a result if the values of either of them matches the search term. But the front-end only gets the data of the name field and will apply its filter only against that data.

That means, if you want to find a state based on its code, and you don't include the code values in the options passed to the response such that it is available to sifter (and, of course, if the state's code isn't similar to its name), then you won't find anything.

Another example where I anticipate this could be an issue is full text search and other kinds of similarity searches, because of the normalization applied to the data and the search term that isn't apparent to the front-end.
Though I suppose, you should provide search result highlighting wherever possible so that the user isn't confused about the results - which would then let sifter find the results - it's still odd to have sifter needlessly apply its own filter on top of already filtered results.

Do you see any possibility in removing stifer.js, while initializing tom-select?

Setting searchField to an empty list seems to remove the filtering aspect while retaining the match highlighting: https://jsfiddle.net/o3xshkLc/
Seems like the best of both worlds? I don't have the faintest clue how to set that within the django-formset script though.

[jrief]

Setting searchField to an empty list seems to remove the filtering aspect while retaining the match highlighting

That could be achieved easily by adding an extra attribute to the Selectize widget. Any proposal how to name this?

[Actionb]

Wouldn't it be better to just generally set searchField to an empty list if a search_lookup is provided to the widget? Because when search_lookup is set, IncompleteSelectResponseMixin._fetch_options already does the filtering and sifter's second filtering isn't ever needed. Or am I missing something?

[jrief]

Wouldn't it be better to just generally set searchField to an empty list if a search_lookup is provided to the widget?

I will try and check if the UX remains the same.

How important is this for you now, or can this feature wait a little bit?

[Actionb]

I can wait.

Is there a way to re-initialize an 'selectize' element with the appropriate searchField value? Doesn't have to be pretty, just a bit of javascript that I can slap into a script block to test things out perhaps? It seems like such a small thing, but I can't figure out how to do it for django-formset.

[jrief]

Is there a way to re-initialize an 'selectize' element with the appropriate searchField value?

This has been implemented in version 1.5. Please try it out.