cli icon indicating copy to clipboard operation
cli copied to clipboard

`sf flow run test` does not return a custom error message

Open ChristianMenzinger opened this issue 2 months ago • 4 comments

Summary

If you execute a Flow Test with Salesforce CLI command sf flow run test it does not return a custom error message for a failed FlowTest, but it is displayed in the FlowTest UI

Steps To Reproduce

General Steps: Create a Flow Test for a record triggered Flow Create an assertion which will fail and define a custom error message in the assertion Run the Flow Test in the UI - inspect errors with "Run Test and View Details" next to the Flow Test Table row - for the failure you will the custom error message Run the same Flow Test from the CLI with output format JSON -> "message" will be empty

Sample Metadata to Reproduce

Clone repo https://github.com/ChristianMenzinger/flow-test-metadata Create a new scratch org Push all metadata to new scratch org Open Org, Open Flow Builder for Flow "Check Escalated in Cases" Open "View Tests" Click chevron on the test row and select "Run Test and View Details" Inspect the failed assertion:

The assertion failed because a condition was not met. Condition {!$Record.IsEscalated} Equals true Custom failure message Escalated was not set to true. Condition evaluation {!$Record.IsEscalated} (false) Equals true

Notice the custom failure message "Escalated was not set to true"

Run the same test with the CLI command: sf flow run test -t Check_Escalated_in_Cases.Check_Escalated_in_Cases_Created_Status_is_Escalated_Escalated_is_False -r json -y

Inspect the output:

{
  "status": 100,
  "result": {
    "summary": {
      "failRate": "100%",
      "failing": 1,
      "hostname": "https://customization-speed-3216-dev-ed.scratch.my.salesforce.com",
      "orgId": "00DKO000000NIzq2AG",
      "outcome": "Failed",
      "passRate": "0%",
      "passing": 0,
      "skipped": 0,
      "testRunId": "",
      "testStartTime": "2025-11-05T14:52:16.731Z",
      "testsRan": 1,
      "username": "[email protected]",
      "commandTime": "1 ms",
      "testExecutionTime": "276 ms",
      "testTotalTime": "276 ms"
    },
    "tests": [
      {
        "Id": "",
        "QueueItemId": "",
        "StackTrace": "",
        "Message": "",
        "AsyncApexJobId": "",
        "MethodName": "Created: Status is Escalated, Escalated is False",
        "Outcome": "Fail",
        "ApexClass": {
          "Id": "07MKO000003z7am2AA",
          "Name": "Check_Escalated_in_Cases",
          "NamespacePrefix": "flowtesting"
        },
        "RunTime": 42,
        "FullName": "flowtesting__Check_Escalated_in_Cases.Created: Status is Escalated, Escalated is False"
      }
    ]
  }
}

Notice "Message" is empty

Expected result

"Message" should display the custom error message as in the Flow Test UI

Actual result

"Message" is empty

System Information

{
  "architecture": "darwin-arm64",
  "cliVersion": "@salesforce/cli/2.110.22",
  "nodeVersion": "node-v22.21.0",
  "osVersion": "Darwin 24.6.0",
  "rootPath": "/Users/christianmenzinger/.local/share/sf/client/2.110.22-7013be3",
  "shell": "zsh",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.2.37 (core)",
    "@oclif/plugin-commands 4.1.35 (core)",
    "@oclif/plugin-help 6.2.33 (core)",
    "@oclif/plugin-not-found 3.2.70 (core)",
    "@oclif/plugin-plugins 5.4.50 (core)",
    "@oclif/plugin-search 1.2.33 (core)",
    "@oclif/plugin-update 4.7.11 (core)",
    "@oclif/plugin-version 2.2.34 (core)",
    "@oclif/plugin-warn-if-update-available 3.1.50 (core)",
    "@oclif/plugin-which 3.2.41 (core)",
    "@salesforce/cli 2.110.22 (core)",
    "agent 1.24.18 (core)",
    "apex 3.8.3 (core)",
    "api 1.3.3 (core)",
    "auth 3.9.17 (core)",
    "data 4.0.59 (core)",
    "deploy-retrieve 3.23.8 (core)",
    "flow 1.0.2 (user) published 197 days ago (Mon Apr 21 2025)",
    "info 3.4.93 (core)",
    "limits 3.3.67 (core)",
    "marketplace 1.3.8 (core)",
    "org 5.9.38 (core)",
    "packaging 2.22.0 (core)",
    "schema 3.3.82 (core)",
    "settings 2.4.48 (core)",
    "sobject 1.4.76 (core)",
    "telemetry 3.6.61 (core)",
    "templates 56.3.66 (core)",
    "trust 3.7.113 (core)",
    "user 3.6.38 (core)",
    "sfdx-plugin-package-xml 0.0.0-development (link) /Users/christianmenzinger/dev/metacache/sfdx-plugin-package-xml",
    "sfdx-plugin-source-read 1.5.6 (user) published 52 days ago (Sun Sep 14 2025)"
  ]
}

ChristianMenzinger avatar Nov 05 '25 14:11 ChristianMenzinger

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

github-actions[bot] avatar Nov 05 '25 14:11 github-actions[bot]

hey @ChristianMenzinger, sorry for the late reply.

We checked with the Flow team and they confirmed they haven't exposed the flow test error message in the sobjects used to fetch the results yet.

I'll assign this ticket to the team and wait until this gets exposed for their CLI plugin to include the msgs.

cristiand391 avatar Nov 11 '25 19:11 cristiand391

This issue has been linked to a new work item: W-20192625

git2gus[bot] avatar Nov 11 '25 19:11 git2gus[bot]

We have determined that the issue you reported exists in code owned by another team that uses only the official support channels. To ensure that your issue is addressed, open an official Salesforce customer support ticket with a link to this issue. We encourage anyone experiencing this issue to do the same to increase the priority. We will keep this issue open for the community to collaborate on.

github-actions[bot] avatar Nov 11 '25 19:11 github-actions[bot]