dlang
Improve search on code.dlang.org
Description
The search feature on code.dlang.org often fails to return relevant results for the user's query.
- Packages whose names include the search term as a substring are not found.
- A search for the exact name of a package sometimes fails to find it.
What are rough milestones of this project?
Fixing the issues linked above, dlang/dub-registry#93 and dlang/dub-registry#417.
How does this project help the D community?
Improving code.dlang.org's search would make it easier for D programmers to find existing packages to use and/or contribute to.
Recommended skills
Web application development.
Point of Contact
@s-ludwig, @Geod24, @WebFreak001
References
Previous attempts:
- https://github.com/dlang/dub-registry/pull/481
- https://github.com/dlang/dub-registry/pull/497
I'm also very much interested in improving the search on code.dlang.org. In particular because it doesn't do sub-package searches and my project giD has all of its library bindings as sub-packages.
With how bad the search feature is, I think D developers are truly missing out on potential libraries which already have D implementations and prospective D developers could even be getting turned away because they don't see support for a library they want to use (such as Gtk 4).
Has there been any other activity with efforts to improve things?
After looking at the dub-registry code and the fact it uses MongoDB, I am considering writing a standalone search engine as a service. It could mirror the package data and provide a REST API which could be embedded in the official site and have it's own web page as well.
I’d just use an established search engine/database for this, e.g. meilisearch. Sounds more maintainable than rolling your own.
@pbackus check this PR and see if that is what you had in mind: https://github.com/dlang/dub-registry/pull/600
@dejlek From the description, it sounds like it fixes dlang/dub-registry#93, but I don't understand the code well enough to tell if it solves dlang/dub-registry#417. Either way, it's a big improvement.
@pbackus Yes, it does resolve dlang/dub-registry#93. In regards to dlang/dub-registry#417. That has to do with the dash being interpreted as a word separator and it then searches with a logic OR which seems fairly useless for this application. Indeed, matplotlib-d has the same results as matplotlib d. Double quoting it "matplotlib-d" as mentioned in #490 in one of the comments does appear to search for the exact match.
It doesn't seem like allowing special characters in this search feature makes sense though, so I pushed an additional commit to my PR which escapes special characters (backslashes and double-quotes) and then double quotes the string when doing the FTS search, so that any value entered is taken as the literal search term.
