freesound icon indicating copy to clipboard operation
freesound copied to clipboard

Not possible to use negation operator in search filter

Open ffont opened this issue 10 months ago • 12 comments

It is currently not possible to use negation operator in search filters because our code in search_filter_make_intersection will add + signs to the filter terms and that results in a syntax like -+tag:guitar which is not valid for Solr. We should update the method to allow for this type of filtering and also add tests for that. This is specially relevant in the API, where some users try to do more advanced filters.

ffont avatar Mar 07 '25 12:03 ffont

Hello, @ffont. I would like to work on this bug. I would be a first time contributor, and looking at the code this seems like it would be a good starter issue for me to begin to get familiar with the codebase.

I'll wait for you to assign this issue to me before making a PR, but in the meantime I'll get familiar with the Solr query docs.

trevorspecht avatar Mar 13 '25 18:03 trevorspecht

Hi Trevor, thanks for your message and for offering to contribute. I think this issue probably has a rather easy fix, but it requires some rather advanced knowledge about how Freesound works, how it communicates to Solr and how search engine tests should be run. There is some information in our DEVELOPERS file. The summary is that we pre-process search filters before sending them to Solr, and some time ago we introduced a change in which we would add "+" sign before each filter term (because Solr changed some default behavior in a backwards incompatible way). This code we made does not work well when a filter term already has the "-" sign on it. In that case we should not add the "+". You could start setting up the local Freesound installation and trying to run the search engine tests. Then add a test which uses a filter with a negation operator. That test should fail. It might be a big hard to follow the code to find where these "+" signs are added, let me know if you need assistance. Thanks!

ffont avatar Mar 14 '25 09:03 ffont

Thank you Frederic, and I appreciate the offer for help. I'll let you know if I reach a point where I need it.

One thing I found when going through the readme development setup instructions is that one file it says to download from Google is missing: Freesound development similarity index

I don't know how important that is, but I'll keep going without it for now.

trevorspecht avatar Mar 14 '25 16:03 trevorspecht

OK sorry another issue when following the readme instructions

When I run docker compose build

I get failed to resolve source metadata for docker.io/library/freesound:2025-03: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

trevorspecht avatar Mar 14 '25 18:03 trevorspecht

Hi @trevorspecht Did the following step run successfully for you?

  1. Build the base Freesound Docker image

    make -C docker
    

Bomme avatar Mar 16 '25 13:03 Bomme

@Bomme yes that step worked for me.

trevorspecht avatar Mar 16 '25 14:03 trevorspecht

I think I found the source of confusion and just made a fix on master.

Explanation: On executing make -C docker it would only build the py2 image because that was chosen by Make if no target is explicitly defined. Before all was chosen as the target which would build both py2 and py3. With the latest commit it should work again.

Bomme avatar Mar 16 '25 16:03 Bomme

@Bomme thank you, this fixed the problem!

trevorspecht avatar Mar 17 '25 01:03 trevorspecht

@Bomme sorry to keep tagging you. I am able to run everything locally now and I have made a local branch and added a failing test that will stop failing once I fix the query filter negation issue.

I was going to push my commit and create a draft PR, but I don't have push permissions. Is there another way to do this or do I need permissions to open a PR?

trevorspecht avatar Mar 21 '25 02:03 trevorspecht

@trevorspecht no worries at all :) Please continue asking if you have any more questions.

I think the standard way would be that you fork the repository and push your changes to your repository/branch. Once the code is ready you can open a PR on the main repository with the changes from your fork.

https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project

Bomme avatar Mar 21 '25 07:03 Bomme

PR is complete and ready for review

trevorspecht avatar Apr 03 '25 22:04 trevorspecht

PR has been approved and merged. thanks for your guidance @ffont!

trevorspecht avatar Apr 11 '25 23:04 trevorspecht