App icon indicating copy to clipboard operation
App copied to clipboard

[$250] [Explain feature] Add support for the "Explain" button in the front end

Open marcaaron opened this issue 3 weeks ago • 6 comments

Part of the "Explain Concierge AI actions" project

Main issue: https://github.com/Expensify/Expensify/issues/567869

Feature Description

  • For this feature we need to implement an "Explain" button in the frontend for any report action that has a reasoning field.
  • This button will exist in the right-click menu for now
  • To start it will be visible on the following report actions:
    • Concierge auto-categorized something using AI
    • Concierge auto-submitted a report
  • Not shown in mockup: When the user taps explain we will create a comment thread off of the system message
  • There should be a comment from the user's perspective that says: "Please explain this to me."
  • Concierge should then respond with reason.

BE Changes are in development now.

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~022004286620315057042
  • Upwork Job ID: 2004286620315057042
  • Last Price Increase: 2025-12-25
Issue OwnerCurrent Issue Owner: @eh2077

marcaaron avatar Dec 25 '25 20:12 marcaaron

Job added to Upwork: https://www.upwork.com/jobs/~022004286620315057042

melvin-bot[bot] avatar Dec 25 '25 20:12 melvin-bot[bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @eh2077 (External)

melvin-bot[bot] avatar Dec 25 '25 20:12 melvin-bot[bot]

Auto-assign attempt failed, all eligible assignees are OOO.

melvin-bot[bot] avatar Dec 25 '25 20:12 melvin-bot[bot]

Auto-assign attempt failed, all eligible assignees are OOO.

melvin-bot[bot] avatar Dec 25 '25 20:12 melvin-bot[bot]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Add support for the "Explain" button in the front end

What is the root cause of that problem?

New feat

What changes do you think we should make in order to solve the problem?

  1. In the report action context menu, check if the report action has a reasoning field:
const hasExplanation = !isEmptyObject(reportAction?.message?.[0]?.reasoning);

if (hasExplanation && (reportAction?.actionName === CONST.REPORT.ACTIONS.TYPE.CONCIERGE_AUTO_CATEGORIZED || 
    reportAction?.actionName === CONST.REPORT.ACTIONS.TYPE.CONCIERGE_AUTO_SUBMITTED)) {
    contextMenuOptions.push({
        icon: Expensicons.QuestionMark,
        text: 'Explain',
        onPress: () => handleExplainAction(reportAction),
    });
}
  1. Create the explanation thread

function handleExplainAction(reportAction: ReportAction) {
    const threadReportID = reportAction.childReportID || createThreadReport(reportAction);    
    // Add user's question comment
    addComment(threadReportID, "Please explain this to me.");
    
    Navigation.navigate(ROUTES.REPORT_WITH_ID.getRoute(threadReportID));
}
  1. Add new action types in const
CONST.REPORT.ACTIONS.TYPE.CONCIERGE_AUTO_CATEGORIZED = 'CONCIERGE_AUTO_CATEGORIZED';
CONST.REPORT.ACTIONS.TYPE.CONCIERGE_AUTO_SUBMITTED = 'CONCIERGE_AUTO_SUBMITTED';

What alternative solutions did you explore? (Optional)

Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.

samranahm avatar Dec 25 '25 20:12 samranahm

⚠️ @samranahm Thanks for your proposal. Please update it to follow the proposal template, as proposals are only reviewed if they follow that format (note the mandatory sections).

github-actions[bot] avatar Dec 25 '25 20:12 github-actions[bot]

Proposal

Please re-state the problem that we are trying to solve in this issue.

We need to implement an "Explain" button in the frontend context menu (right-click/long-press) for specific Concierge AI report actions. When triggered, the app should create a comment thread where the user asks for an explanation, and the Concierge responds with the pre-existing reasoning data.

What is the root cause of that problem?

This is a new feature implementation. The frontend currently does not have a menu option to surface the reasoning data stored in AI-generated report actions, nor does it have the logic to initiate an explanation thread from the context menu.

What changes do you think we should make in order to solve the problem?

We should make the following changes:

  • CONST.ts: Add CONCIERGE_AUTO_CATEGORIZED and CONCIERGE_AUTO_SUBMITTED to REPORT.ACTIONS.TYPE. Add EXPLAIN_ACTION to SENTRY_LABEL.CONTEXT_MENU.

  • Translation files: Add reportActionContextMenu.explain to src/languages/en.ts (and other language files) to provide the button label.

  • ContextMenuActions.ts: Add a new action object to the CONTEXT_MENU_TYPES.REPORT_ACTION group.

  • The shouldShow property will check if the reportAction has a reasoning field and matches the specified Concierge action types.

  • The onPress property will call a new action Report.explainAction().

  • Report.ts (Actions): Create a new action explainAction(reportID, reportAction). This function will:

  • Identify or create a thread for the report action using ReportUtils.getOrCreateThread.

  • Use addComment to post the localized string "Please explain this to me." into that thread.

  • Navigate the user to the thread immediately using Navigation.navigate.

What alternative solutions did you explore? (Optional)

We can considered adding the button directly to the report action item UI, but the requirements specify the right-click menu to keep the main chat interface clean. Using ContextMenuActions.ts is the preferred approach as it automatically handles the UI for Web, Desktop, and Mobile (BottomDockedActionPicker) through the same configuration object.

beodw avatar Dec 25 '25 22:12 beodw

📣 @samranahm 🎉 An offer has been automatically sent to your Upwork account for the Contributor role 🎉 Thanks for contributing to the Expensify app!

Offer link Upwork job Please accept the offer and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑‍💻 Keep in mind: Code of Conduct | Contributing 📖

melvin-bot[bot] avatar Dec 26 '25 05:12 melvin-bot[bot]

Thanks @samranahm! Let me know if you have any questions? The backend should be deployed hopefully next week.

For auto-submit I think the actions can only be:

ACTION_SUBMITTED

function handleExplainAction(reportAction: ReportAction) {
    const threadReportID = reportAction.childReportID || createThreadReport(reportAction);    
    // Add user's question comment
    addComment(threadReportID, "Please explain this to me.");
    
    Navigation.navigate(ROUTES.REPORT_WITH_ID.getRoute(threadReportID));
}

I think this is not how threads are created. Check the examples in the codebase. Also, let's just call this function explain()

marcaaron avatar Dec 26 '25 05:12 marcaaron

@eh2077 @marcaaron PR #78487 ready for review.

samranahm avatar Dec 27 '25 00:12 samranahm