SwiftPackageIndex
Keyword search tweak
Rebased version of #1905, to kick off CI and have an Xcode PR view open.
To do:
- [x] Review/merge review changes #1955
- [ ] Re-run/review query perf tests after merging #1955
The current setup fails the performance test, but only but a small amount.
Based on what I found while digging around in pgAdmin and the explanation of the query, the FROM terms that build up the ts_vectors get inlined into the flow, and don't add to the cardinality of the query - but I think we could pre-load that computation into the materialized search view (by adding a migration that builds the vectors for each row), and I was fully expecting we might want to do that prior to merging this.
As a general pattern, I tried to "keep and extend" all the existing queries, enhancing them with the ranking rather than replacing any of the functionality. Figured that would be the best proof of concept path, and we could talk through the existing query structure to refine anything we want to clean up, including the potential bug you highlighted above with the keyword queries.
For what it's worth, all of the keywords (today/currently) are in English - which makes them particularly amenable to the tsvector normalization and ranking process, and while I haven't measured the same metrics for the keyword results (precision, recall, etc) - they're noticeable better with a mechanism using the tsquery @@ tsvector WHERE comparison. There wasn't really an impact to adding ranking to that set though (the keyword query) - it doesn't flow to impacting the end result search query all that much.
I suspect there's not much to do WRT performance tuning after we merge #1955. Some of the deltas should go away and the explain analyze I ran on the package subquery showed only very, very minor changes. It's probably not even worth touching the materialised view for that - but let's see once we've merged #1955.
I'm puzzled by why percy is flagging visual changes for this PR. There are no html changes. The diff in percy shows something in the footer which looks like aliasing issues. Any idea what this is about, @daveverwer ?
Ok, rebase fixed that. I guess there was interference due to changes on main which we didn't have in this branch yet.