sdk-php icon indicating copy to clipboard operation
sdk-php copied to clipboard

[Question] Sentry Integration for error logging

Open khoerintus opened this issue 4 years ago • 11 comments

Is there any best practice to integrate temporal with sentry for error logging? I know that I can use sentry php sdk, try catch in my code, and send the error. But is there any global error handler that catch every error (from Workflow or Activity), so I can just send the error from there? Or I can integrate roadrunner with sentry?

khoerintus avatar Apr 20 '21 15:04 khoerintus

Good question. We did not provide a separate interceptor for errors, so you will have to override the transport to handle them, like:

use Spiral\RoadRunner\Worker;
use Temporal\Worker\Transport\HostConnectionInterface;
use Temporal\WorkerFactory;

$worker = WorkerFactory::create()
    ->newWorker();

/// ...

$worker->run(new class implements HostConnectionInterface {
    public function error(\Throwable $e) { ... } // Intercept errors
});

I agree that this functionality should be added. But while it is not there, you will have to write your own transport decorator.

SerafimArts avatar Apr 20 '21 15:04 SerafimArts

Thanks @SerafimArts for the answer. I will try it later.

khoerintus avatar Apr 20 '21 15:04 khoerintus

Hi @SerafimArts , I've tried to write my own transport decorator and use it. But the error function is never called.

I found that Exception thrown in Workflow class, is caught in Temporal\Internal\Workflow\Process\start(), and Exception thrown in Activity class, is caught in Temporal\Internal\Transport\Router\handle(). If I put the code to send the error to sentry in those functions, it sent successfully.

khoerintus avatar Apr 22 '21 12:04 khoerintus

Yes, that's right. The example I showed only intercepts fatal SDK errors (e.g. errors connecting to the server), and all "business logic" errors inside Workflows and Activities are considered "normal work" and sent directly to the Temporal.

Hmmm... @mfateev perhaps it makes sense to add APM (Sentry, New Relic, Kibana, etc) support to the Temporal Server? This will allow to nicely structure such errors for all SDKs at once.

SerafimArts avatar Apr 22 '21 15:04 SerafimArts

Meanwhile, what is the current solution if we want to use sentry for temporal and php sdk ?

changwuf31 avatar Apr 26 '21 15:04 changwuf31

None. We are still thinking how would that look like. Most likely we can provide you hook entry point for activities, but not for the workflows as I don't think Sentry SDK can work in async mode.

wolfy-j avatar Apr 26 '21 15:04 wolfy-j

I ran into the same issue, except I have multiple handlers in monolog (file log + Sentry). Doing a try-catch with logging & re-throwing inside an activity is not very convenient.

The proposal document mentions the idea of injecting a PSR-3-compatible logger into a worker: https://github.com/temporalio/proposals/blob/master/php/php-rr-sdk.md#logging Was it rejected or it's just not implemented yet?

kozlice avatar Jul 05 '21 15:07 kozlice

Not implemented due to the fact that you create activity classes manually, so you control injections to them.

The container refactor planned later this year, with some internal changes and optimizations, including the ability to intercept activity call (for exception catching or state resets).

P.S. Proposal does not cover all the functions from 1.0 of SDK and has some extra features kept for the later day. It was not 1-to-1 architecture plan.

wolfy-j avatar Jul 05 '21 15:07 wolfy-j

You can find an example of integration here: https://github.com/spiral/temporal-bridge/blob/master/src/InvokeActivityRouter.php

Essentially you can intercept the activity call and manipulate it with input or output parameters, or handle exceptions.

wolfy-j avatar Feb 19 '22 16:02 wolfy-j

@wolfy-j :+1: Superb, however it will be much better, if some of the sdk properties are made protected, so we can simply override the necessary functions.

changwuf31 avatar Mar 10 '22 10:03 changwuf31

The link above is broken as that functionality was removed from temporal-bridge. Anyway, here's how it looked like. Though fully copying the whole WorkerFactory instead of just extending is redundant as for me. The only reason was to access $this->queues which is private. Accessing parent's private field is tricky, I've embraced this library.

And yes, this is really-really hacky, "don't do that at home".

BTW You may ask why was that functionality removed from temporal-bridge. The reason is introducing "finalizers" functionality which is enough for example for cleaning resources to avoid memory leaks but not enough for error handling as finalizers are working similar to final block after try-catch. So You cannot access errors in finalizers.

programmador avatar Sep 07 '22 09:09 programmador