[Feature-Request] Enhance API for better Syncing of Frame/Configurations

[imwhocodes]

trafficstars

why:

On more complex pipelines (ex multiple stage of NN intervaled with nodes of image manipulation and spatial calculation) keeping the frame in sync can be a real pain with the current interface, the only present way is to put ScriptNodes in between the slow-path and the point of merge to manually sync the frame again inside the pipeline Also there is no way of knowing which *Config (ex ImageManipConfig) is related to which frame after the generation of it, creating the need to manually marshalling data to be able to sync a crop region and it's configuration

what:

I'm proposing 3 small change to the API, 2 of them are independent (and easy to implement) but can work in synergy to enhance a lot the quality of life during development of advanced pipeline without being a breaking change

1. All type of messages should have the sequenceNumber field (same as ImgFrame) and user configurable (imho it should be a propriety of Buffer itself)
2. Nodes that take a *Frame and a *Config should have a counterpart/extension of .setWaitForConfigInput(True) (something on the like of .setProcessSynced(True)) which if set to True it consume 1 form the *Config input queue and then consume from the *Frame input queue until the sequence number are aligned
3. All inputs of all node should have a passthrough counterpart on the outputs of the node itself

how:

Here pseudo-python of the possible internals of ImageManip node putting all 3 points from before together:

config = initialConfig

# main node loop
while True: 

    img = queue_inputImage.get()
    img_seqn = img.getSequenceNum()

    if getWaitForConfigInput() or getProcessSynced():

        config = queue_inputConfig.get()
        
        if getProcessSynced():

            config_seqn = config.getSequenceNum()

            while img_seqn < config_seqn:
                img = queue_inputImage.get()
                img_seqn = img.getSequenceNum()

            if img_seqn != config_seqn:
                node.warn(f"Frame were lost!\t{img_seqn} > {config_seqn}\tSkipping!")
                node.debug(f"Try increasing queue length using ImageManip::inputImage.setQueueSize({ ( img_seqn - config_seqn ) + 1 })")
                continue #GO BACK TO START OF WHILE LOOP

    else:

        try_msg_config = queue_inputConfig.tryGet()

        if try_msg_config is not None:
            config = try_msg_config



    # Do normal image manipulation 

    # Send resulting manipulated image on the out queue (WITH SEQUENCE NUMBER SAME AS img_seqn)

    queue_passthroughConfig.send(config)
    queue_passthroughImage.send(img)