[$250] Search - Inconsistency between No tag and No category filters

[lanitochka17]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Version Number: 9.0.56-0 Reproducible in staging?: Y Reproducible in production?: Y If this was caught on HybridApp, is this reproducible on New Expensify Standalone?: N/A If this was caught during regression testing, add the test name, ID and link from TestRail: N/A Email or phone of affected tester (no customers): [email protected] Issue reported by: Applause - Internal Team

Action Performed:

Peconditions: existing account with workspace created, tags are enabled for the workspace and the following expenses are created:

• at least one expense with category selected and no tag
• at least one expense with tag selected and no category
• at least one expense with no tag and no category

1. Log in with account from preconditions
2. Click on Search icon in the bottom
3. Click on Filters, apply filters for 'No category', the tag from preconditions and view results
4. Verify that only expense matching both filters is displayed
5. Click on Filters, apply filters for the category from preconditions and 'No tag'
6. Verify expenses displayed

Expected Result:

Only expenses matching category selected in filters and without tags should be displayed

Actual Result:

All expenses without tags are displayed, category filter is omitted

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• [ ] Android: Standalone
• [ ] Android: HybridApp
• [ ] Android: mWeb Chrome
• [ ] iOS: Standalone
• [ ] iOS: HybridApp
• [ ] iOS: mWeb Safari
• [x] MacOS: Chrome / Safari
• [ ] MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

https://github.com/user-attachments/assets/84878378-831e-42b5-ad1f-d4e56eadf76b

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~021853546282413139923
• Upwork Job ID: 1853546282413139923
• Last Price Increase: 2024-11-04

Issue Owner

Current Issue Owner: @jayeshmangwani

[melvin-bot[bot]]

Triggered auto assignment to @slafortune (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.

[lanitochka17]

@slafortune FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

[lanitochka17]

We think that this bug might be related to #wave-control

[melvin-bot[bot]]

@slafortune Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

[slafortune]

Either way - you can be left with options that don't pertain if you filter by tags then categories. While it doesn't have a category, it also doesn't have the proper tag to be included in the filter -

[melvin-bot[bot]]

Job added to Upwork: https://www.upwork.com/jobs/~021853546282413139923

[melvin-bot[bot]]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @jayeshmangwani (External)

[klajdipaja]

This is a BE issue. The query that we send seems to me like it has the right data in it. This is what we send to BE for a search wiht no tag but Advertising category type:expense status:all category:Advertising tag:none :

{
    "type": "expense",
    "status": "all",
    "sortBy": "date",
    "sortOrder": "desc",
    "filters": {
        "operator": "and",
        "left": {
            "operator": "eq",
            "left": "category",
            "right": "Advertising"
        },
        "right": {
            "operator": "eq",
            "left": "tag",
            "right": "none"
        }
    },
    "inputQuery": "sortBy:date sortOrder:desc type:expense status:all category:Advertising tag:none",
    "hash": 113990055,
    "offset": 0
}

Maybe we need to provide a different value instead of none when No Tag or No Category is selected?

[ijmalik]

Proposal

Please re-state the problem that we are trying to solve in this issue. In Search, there is an inconsistency in the behavior when selecting "No Tag" or "No Category" filters.

What is the root cause of that problem?

1- https://github.com/Expensify/App/blob/ad0f834b805b763fb5885d2003ca2bdd6e33ff64/src/pages/Search/SearchAdvancedFiltersPage/SearchFiltersCategoryPage.tsx#L20-L24 Initially, searchAdvancedFiltersForm returns "none" for the category when the user selects "No Category."

2- https://github.com/Expensify/App/blob/afa7b33107747d0242acfd01d032d5339be390d9/src/libs/actions/Search.ts#L139-L141 The issue then arises due to the way jsonQuery is structured when passed to the SEARCH API. For example, the current jsonQuery looks like this:

{
    "type": "expense",
    "status": "all",
    "sortBy": "date",
    "sortOrder": "desc",
    "filters": {
        "operator": "and",
        "left": {
            "operator": "eq",
            "left": "category",
            "right": "Fees"
        },
        "right": {
            "operator": "eq",
            "left": "tag",
            "right": "none"
        }
    },
    "inputQuery": "sortBy:date sortOrder:desc type:expense status:all category:Fees tag:none",
    "hash": 46304472,
    "recentSearchHash": 2055726720,
    "offset": 0
}

The backend does not interpret "none" consistently, whether it appears in filters or inputQuery. Additionally, filter order affects the results.

Correct Results occur, for example, when "No Tag" is selected before "Fees" in the category:

type:expense status:all tag:none category:Fees
type:expense status:all category:none tag:"First Tag"
type:expense status:all tag:none merchant:m10

Incorrect Results are returned, for example, when "Fees" is selected before "No Tag":

type:expense status:all category:Fees tag:none
type:expense status:all tag:"First Tag" category:none
type:expense status:all merchant:m10 tag:none

This is the root cause of the inconsistent behavior with "No Tag" and "No Category" filters.

What changes do you think we should make in order to solve the problem? https://github.com/Expensify/App/blob/afa7b33107747d0242acfd01d032d5339be390d9/src/libs/actions/Search.ts#L139 Replace with

const jsonQuery = JSON.stringify(removeNoneFromTagAndCategory(queryWithOffset));

and add the following function to replace "none" with empty string

function removeNoneFromTagAndCategory(queryObject) {
    if (queryObject.filters?.right) {
        const { left, right } = queryObject.filters.right;
        if ((left === "tag" || left === "category") && right === "none") {
            queryObject.filters.right.right = '';
        }
    }
    
    if (queryObject.inputQuery) {
        queryObject.inputQuery = queryObject.inputQuery
            .replace("tag:none", "tag:")
            .replace("category:none", "category:");
    }
    return queryObject;
}

What alternative solutions did you explore? (Optional) Rather than replacing "none" with an empty string, we could reorder the jsonQuery properly before the API call to ensure consistent results. For example:

type:expense status:all tag:none category:Fees
type:expense status:all category:none tag:"First Tag"
type:expense status:all tag:none merchant:m10

Screenshot:

Videos: Before Change: Correct Results

https://github.com/user-attachments/assets/2020a3cb-1cea-4aa9-ba99-7da481a86472

Before Change: Incorrect Results

https://github.com/user-attachments/assets/20018427-6ea4-48a3-aad4-ad52ef0a1b2c

After Change: All Results are Correct

https://github.com/user-attachments/assets/76785e1b-652a-49e5-b072-5ac7860204c3

[jayeshmangwani]

@ijmalik Issue appears to be coming from the backend, so it seems internal for now. could you please confirm if the issue is from the FE or BE?

[ijmalik]

@jayeshmangwani Hi, This is a front-end (FE) issue, and my proposal above provides the resolution.https://github.com/Expensify/App/issues/51828#issuecomment-2461892145

On Fri, Nov 8, 2024, 11:25 AM Jayesh Mangwani @.***> wrote:

@ijmalik https://github.com/ijmalik Issue appears to be coming from the backend https://github.com/Expensify/App/issues/51828#issuecomment-2456815267, so it seems internal for now. could you please confirm if the issue is from the FE or BE?

— Reply to this email directly, view it on GitHub https://github.com/Expensify/App/issues/51828#issuecomment-2463875779, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQ6GVN7BPUVH6XHQCXNXITZ7RKN3AVCNFSM6AAAAABQ65FZHOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRTHA3TKNZXHE . You are receiving this because you were mentioned.Message ID: @.***>

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Whoops! This issue is 2 days overdue. Let's get this updated quick!

[jayeshmangwani]

@ijmalik I’ve tested filter a few times, and selecting "none" isn’t working correctly and I still think this should be handled on the backend, e.g. why we need to logic like removeNoneFromTagAndCategory in the FE

[ijmalik]

@jayeshmangwani

I agree, it would be more efficient to handle this on the backend instead of having logic like 'removeNoneFromTagAndCategory' in the frontend.

[melvin-bot[bot]]

@slafortune @jayeshmangwani this issue was created 2 weeks ago. Are we close to a solution? Let's make sure we're treating this as a top priority. Don't hesitate to create a thread in #expensify-open-source to align faster in real time. Thanks!

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Huh... This is 4 days overdue. Who can take care of this?

[jayeshmangwani]

Issue has been switched to Internal. @slafortune, we’re waiting for an internal engineer to pick this issue, right?

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Huh... This is 4 days overdue. Who can take care of this?

[melvin-bot[bot]]

@slafortune, @jayeshmangwani 6 days overdue. This is scarier than being forced to listen to Vogon poetry!

[jayeshmangwani]

Internal issue and is currently waiting to be pick

cc: @slafortune

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Huh... This is 4 days overdue. Who can take care of this?

[slafortune]

Yes - internal hot pick - waiting for an engineer to pick this up

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Still overdue 6 days?! Let's take care of this!

[mvtglobally]

Issue not reproducible during KI retests. (First week)

[ijmalik]

Hi @mvtglobally,

The issue is still reproducible in specific cases, particularly when the filter order affects the results.

Incorrect Results are returned, for example, when "Fees" is selected before "No Tag":

type:expense status:all category:Fees tag:none  
type:expense status:all tag:"First Tag" category:none  

Correct Results occur, for example, when "No Tag" is selected before "Fees" in the category:

type:expense status:all tag:none category:Fees  
type:expense status:all category:none tag:"First Tag"  

Let me know if additional details are needed.

Wrong Filter Result:

https://github.com/user-attachments/assets/fa1e9fac-d02f-4d3b-a149-155e8b377464

Correct Filter Result:

https://github.com/user-attachments/assets/78273b2f-bd5b-4b79-986c-7d785f0f0b83

[melvin-bot[bot]]

@slafortune, @jayeshmangwani 10 days overdue. I'm getting more depressed than Marvin.

[jayeshmangwani]

Waiting for an internal engineer to pick this up.

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Eep! 4 days overdue now. Issues have feelings too...

[jayeshmangwani]

Same as this

cc: @slafortune

[melvin-bot[bot]]

@slafortune, @jayeshmangwani Whoops! This issue is 2 days overdue. Let's get this updated quick!