app icon indicating copy to clipboard operation
app copied to clipboard

[Bug] API /reporter/{runId}/testrun Always Returns Success Message Even on Failure

Open seusur opened this issue 10 months ago • 0 comments

The API endpoint https://app.testomat.io/api/reporter/{runId}/testrun always returns a success message {"message":"Test run added"} even when the test result is not actually added to the run. This makes it difficult to detect failures immediately.

To Reproduce

  1. Send a request to https://app.testomat.io/api/reporter/{runId}/testrun with a test result with not correct field value, for example:

    test_results = [{
           "status": "failed",
           "title": "test test test",
           "message": "Some test message",
           "test_id": "{test_id}",
           "example": ["test"],  # not correct field value
       }]
    
    
  2. The API still responds with {"message":"Test run added"}.

  3. However, an error can only be found later in the test case logs, in my example: "Error reporting test to Testomat.io. wrong element type String at 0 (expected array). Check runner logs"

Expected behavior

In response to the request, a clear and concise error message should be returned. For example: { "error": "Error reporting test to Testomat.io. wrong element type String at 0 (expected array). Check runner logs" }

Screenshots

Image

seusur avatar Mar 14 '25 13:03 seusur