simonw
BM25 ranking with search
What's the recommended way of performing BM25 ranking search with sqlite-utils? I found a mention of rank_bm25() in the docs, but I couldn't figure out how to use the function.
Or maybe I'm misunderstanding the docs, and rank_bm25 is actually what sqlite uses behind the scenes when I use full text search (assuming fts_version="FTS4")?
Ok, I dug a bit into sqlite fts5 docs, and apparently it's already using bm25 even in fts5:
... in a full-text query, column rank contains by default the same value as would be returned by executing the bm25() auxiliary function with no trailing arguments. The difference between reading from the rank column and using the bm25() function directly within the query is only significant when sorting by the returned value. In this case, using "rank" is faster than using bm25().