sdk-go
sdk-go copied to clipboard
temporalio
Allow custom time source
Is your feature request related to a problem? Please describe.
My workflow's behavior is heavily time dependent, and I need to simulate how it will behave over dynamically sized time spans in the past and the future. To do so, I need the ability to configure Temporal to behave as if the current time is one dictated by a custom clock.
Describe the solution you'd like
Specifically I’d like to make sure that temporal_workflow.Now(ctx) returns values set by a fake clock I control, and that workflow.Sleep also uses the fake clock.
Additional context
For my full workflow, I need the ability to set the clock back to an arbitrary date, and then to have it fast-forward through workflow sleeps such that calls to get the current time return a result that look as if the sleeps took their full amount of time, but fast-forwarded so that a human can run a many-day-long process in a matter of seconds.
@mfateev suggested I open an issue in response to my question on this topic at https://community.temporal.io/t/custom-time-source-with-the-go-sdk/6174.
This is already present via TestWorkflowEnvironment.SetStartTime.
@mnussbaum - To clarify, while you can set the initial time, the test suite will automatically skip time for you until the next event. So sleeps would resolve immediately (assuming nothing outstanding is also waiting for a shorter period). To set something to happen a specific time after the workflow starts, you can use TestWorkflowEnvironment.RegisterDelayedCallback. See the Godoc on that call. You should set the start time and delayed callbacks before starting the workflow.
@cretz I'd like to use this functionality outside of unit tests, in my operationally deployed software. I was under the impression that TestWorkflowEnvironment didn't connect to a real temporal instance, and thus wasn't suitable for this purpose, is that right?
My use case is that I have application users who want to perform "back testing", ie they want to test the performance of various different operational parameters of the software against historical data. The time period and parameters we'll be back testing over are driven by non-technical user input. I want to expose a user interface in front of my deployed software that lets the end user choose the time period for back testing, which the software then uses to set the Temporal clock and initiate a workflow
I'd like to use this functionality outside of unit tests, in my operationally deployed software.
@mnussbaum - We provide no way to create a workflow.Context outside of a real worker or a test environment. That context contains many things to make workflows function. If you want to do an integration test against a real server, that works, but you don't get time skipping/management. I am afraid time-skipping unit tests or real-server actual time are all we offer in Go SDK.
I was under the impression that TestWorkflowEnvironment didn't connect to a real temporal instance, and thus wasn't suitable for this purpose, is that right?
Suitable for which purpose? It should be suitable for all workflow testing needs.
My use case is that I have application users who want to perform "back testing", ie they want to test the performance of various different operational parameters of the software against historical data. The time period and parameters we'll be back testing over are driven by non-technical user input. I want to expose a user interface in front of my deployed software that lets the end user choose the time period for back testing, which the software then uses to set the Temporal clock and initiate a workflow
If this is on an already-run workflow, you could use the replayer. But that's often only for catching errors, it won't run any activities and you can't really customize it.
Technically you can use the TestWorkflowEnvironment to test things outside of a unit test. You will get to control time and activities and other things you would normally control in a unit test. It sounds like this may be what you need since you are wanting to control the environment, even if we do call it a "test" environment. You get the added benefit of time skipping, so besides setting the start time, it will automatically skip to the next event (be it a timer or whatever). If there are any issues with using the test workflow environment for your use case, let us know and we may be able to alter it.
If you want to do an integration test against a real server, that works, but you don't get time skipping/management. I am afraid time-skipping unit tests or real-server actual time are all we offer in Go SDK.
Yeah, the workflows we're building aren't for integration testing purposes even. I need to be able to manage time in my production deployed workflows, exposed to end users and integrated with the Temporal server. I think that's the heart of this feature request, the ability to control the workflow clock when running workflows with a real Temporal server.
If this is on an already-run workflow, you could use the replayer. But that's often only for catching errors, it won't run any activities and you can't really customize it.
Unfortunately these workflows won't be replays of the past, they'll be first-run with a historical perspective of time.
Technically you can use the TestWorkflowEnvironment to test things outside of a unit test.
If we use the TestWorkflowEnvironment like you're imagining, it wouldn't be connected to a real Temporal-server, right? So the ability to schedule activities and child-workflows on a pool of workers would not be available, and we wouldn't be able to resume a suspended workflow on a different worker then started it, right? Those would be the critical requirements for the scenarios we need to modify our clock in.
I need to be able to manage time in my production deployed workflows, exposed to end users and integrated with the Temporal server.
I am afraid this is not a feature Temporal supports currently. We technically have a time-skipping Temporal server (written in Java, but we compile with Graal for use in our other SDKs as a separate time-skipping binary). However, that is also not a real Temporal server.
If we use the TestWorkflowEnvironment like you're imagining, it wouldn't be connected to a real Temporal-server, right? So the ability to schedule activities and child-workflows on a pool of workers would not be available, and we wouldn't be able to resume a suspended workflow on a different worker then started it, right? Those would be the critical requirements for the scenarios we need to modify our clock in.
Right. Even our time-skipping server may lack some of the features you'd want in that time-skipping is global so it's not suitable for normal server use.
If you can identify exactly what calls you want to intercept to provide mock time, you might be able to use interceptors for this. Every call to workflow.Now() or workflow.Sleep() is interceptable via a WorkflowOutboundInterceptor and you could do whatever you wanted in there. But it isn't necessarily global time management because timeouts and other things wouldn't apply.
If you need to manage time on a production server, I am afraid this is not available. If you need to intercept certain calls you can. Otherwise, if at all possible, you'll need to solve this in user land (i.e. use your own deterministic mock clock and make your own clock invocations when you need time), but you won't get all of the server-specific timing features.
Out of curiosity, can you clarify your use case a bit more? What do you want to mock with this clock? Just workflow.Now() since that is an exact time, or durations like sleep/timer also? Does this include activities? Does this include timeouts? Can you be specific on exactly what you want to use the custom time source for?
Necroing this old issue:
We've got a use case where we want a mocked/manual clock so that we can control when each step of our mock workflow executes. This would allow us to write tests that assert expectations at various points during the workflow execution as well as verify the expected scheduling behavior by manipulating the clock before and after scheduling.
Are you referring to the testsuite? Time automatically skips there, and you can RegisterDelayedCallback to do certain things at certain times. That is the only real environment where you can control time, controlling time in a non-test-environment workflow is unreasonable as many internal things such as timeouts rely on clocks you cannot substitute (e.g. server-side timings).
Yeah, I was. Somehow in digging through the client I missed RegisterDelayedCallback. That does seem to do the trick - I was just expecting/hoping for a power-user mode that allowed for full control of the fake clock.
:+1: Yeah the delayed callback should be enough to do all that is necessary at certain times without having to manually progress a clock yourself.
(closing issue because of the previous staleness and the acknowledged delayed callback approach, but feel free to continue discussing here or in #sdk-go channel on https://t.mp/slack)