SeriesGuide icon indicating copy to clipboard operation
SeriesGuide copied to clipboard

Check if episode search can look for lower-case + upper-case at the same time

Open UweTrottmann opened this issue 5 years ago • 4 comments

E.g. searching for Ü is not the same as ü and other umlauts. I highly suspect this is dependent on the localization setting of the SQLite database.

UweTrottmann avatar Sep 14 '19 11:09 UweTrottmann

Ich habe das Problem gerade auch festgestellt (bei "Mayday" habe ich nach "überlebend" gesucht und die Episode "Ein Überlebender" wurde nicht gefunden; erst, als ich das Ü groß geschrieben habe).

Die Suche ist an sich ja case-insensitive. Schön wäre es, wenn das auch für Umlaute gelten würde.

LG aus Berlin und danke für die tolle Arbeit, Oli

unitb-ore avatar Apr 29 '20 11:04 unitb-ore

It may be possible to specify a tokenizer that supports Unicode "case folding": https://www.sqlite.org/fts3.html#tokenizer

Probably something like CREATE VIRTUAL TABLE txt2 USING fts4(tokenize=unicode61 "remove_diacritics=2");.

Version requirement: SQLite version 3.7.13 (2012-06-11) Not sure what Android version this requires. Edit: https://developer.android.com/reference/android/database/sqlite/package-summary says 3.7 since API 11, so should be OK.

Edit: Room does not seem to note an Android version requirement, so probably works on all versions supported by Room? https://developer.android.com/reference/androidx/room/FtsOptions#TOKENIZER_UNICODE61

Edit: it appears 'remove_diacritics='" is not supported, so using default.

Edit: want 'remove_diacritics=0' anyhow to prevent e.g. "ö" matching "o".

UweTrottmann avatar Nov 17 '21 13:11 UweTrottmann

Nice research in edits, @UweTrottmann!

I was also having a similar issue, and it seems remove_diacritics=2 is not supported below 30.

so probably works on all versions supported by Room?

FtsOptions.TOKENIZER_UNICODE61 has a requirement on it, just not in the Javadoc, see source code. It's possible the annotation wasn't there when you checked.

TWiStErRob avatar Oct 21 '23 18:10 TWiStErRob

@TWiStErRob Thanks for the research! Added a comment to the code about that.

Maybe I'll add a version check then.

UweTrottmann avatar Oct 27 '23 11:10 UweTrottmann