The Lambda response cannot be encoded to JSON

[harshal-ringe]

Description: I'm using a docker image with Lambda with the following dependencies to handle SQS jobs.

"php": "8.1"
"bref/bref": "^2.3",
"bref/laravel-bridge": "^1.2",
"laravel/framework": "^v8.83.27"

I'm getting the exception in the job handler when I try to insert a document in MongoDB. Insert document has non-UTF-8 characters and it throws an exception.

"exception": {
  "class": "MongoDB\\Driver\\Exception\\UnexpectedValueException",
  "message": "Detected invalid UTF-8 for field path \"message\": <div>\ufffd\ufffd</div>三崎 雅也 created a new Task</div>",
  "code": 0,
  "file": "/var/task/vendor/mongodb/mongodb/src/Operation/InsertMany.php:139"
}

This exception gets caught in vendor/bref/laravel-bridge/src/Queue/LaravelSqsHandler.php =>process function's try-catch block. It again throws the same caught exception so that Lambda can handle it. This exception has non-UTF-8 characters and it gives me the following exception.

The Lambda response cannot be encoded to JSON.
This error usually happens when you try to return binary content. If you are writing an HTTP application and you want to return a binary HTTP response (like an image, a PDF, etc.), please read this guide: https://bref.sh/docs/runtimes/http.html#binary-requests-and-responses
Here is the original JSON error: 'Malformed UTF-8 characters, possibly incorrectly encoded'

Due to this error, It keeps on retrying the same job till the lambda doesn't timeout. In my case, It called the same handle function 8000 times in the same lambda execution. To resolve the issue, I tried adding BREF_BINARY_RESPONSES: '1' in environment variable but it didn't work. I'm not using API Gateway as this lambda is used to handle SQS jobs.

The issue gets resolved if I remove the below lines src/Queue/LaravelSqsHandler.php file

// Rethrow the exception to let SQS handle it
 throw $e; 

How to reproduce:

Throw the exception from the Job handler with a message body containing non-UTF-8 characters.