Language filtering on the Hub does not work correctly

[saattrupdan]

(I noticed recently that you changed the language filtering GET parameter from filter to language, but unfortunately my problem still persists)

If I search for Danish models then models like this also pop up, which is not a Danish model, but has the da tag (for "direct assessment"). This could be fixed be filtering by the tag-green class rather than a general tag filtering, as I guess is done currently.

Thanks!

[julien-c]

BTW @beurkinger @elishowk this broke a ton of existing URLs, no? Didn't we agree to not change URLs, just the frontend implementation?

[beurkinger]

@julien-c ?filter=da still works, although the recommended way of doing thing is now ?language=da

[beurkinger]

by the way the branch was reviewed by @Pierrci not @elishowk, so Elis is not concerned by this :)

[beurkinger]

This issue is related to the fact that the backend doesn't finely filter each category, but instead checks every possible field in each models for the values present in filter.

The good news is that thanks to the new query parameters, we will now be able to directly target the fields related to each filter (as they are available individually as language, pipeline_tag, etc. params) :)

[elishowk]

OK :-) Let me know if I can help.

[beurkinger]

@elishowk this is definitely a task that could be tackled now, and that would be a great improvement to the filtering system ! we just have to make sure to keep it backward compatible with the old filter param

[osanseviero]

This should be fixed now

[saattrupdan]

@osanseviero Unfortunately not, as can be seen on the following search results:

https://huggingface.co/models?language=da&sort=trending&search=transquest

The siamesetransquest models are not Danish and thus shouldn't show up. They have the da tag, but not the da language tag.