opus icon indicating copy to clipboard operation
opus copied to clipboard

Add "transform" to Link

Open fermuch opened this issue 9 months ago • 0 comments

I'm trying to reuse as much logic as possible in my pipelines. As such, I'm having some problems when trying to work with them.

Let's imagine I have a message with several different kinds of data, for example: the same message has a device position (lat, lng), a state ("is the light turned on?") and a numeric value ("my current voltage is...")

Since the device sends all of those together, I'd need to parse and validate that message in a single pipeline. And, since that data is already in the pipeline, I should be able to send each part of the message to a specific Pipeline to handle those.

For the position: StorePositionPipeline which stores the location and updates the "this is the current position of the device".

For the state: StoreStatePipeline which takes a state and a state name, and stores it.

For the numeric value: StoreNumericValuePipeline and VerifyVoltagePipeline. The first one is the same as StoreStatePipeline, but keeps track of numbers instead of enums. The second one has to do several things: check if the voltage is in the safe zone, if not maybe create an alert if there wasn't an alert already open for that device, ...

I am currently using a step and then calling the pipeline's call/1, since each one of those Pipelines expects its input in a different format.

By adding a transform to Link, I'd be able to write my pipeline as:

link StorePositionPipeline, transform: &extract_latlng/1
link StoreStatePipeline, transform: &extract_light_state/1
...

And that'd be much more readable IMHO than messing with steps just to transform the data and do a call.

fermuch avatar Oct 20 '23 13:10 fermuch