[Feature-Request] Enhance API for better Syncing of Frame/Configurations

[imwhocodes]

I initially posted this here but only now realised the core implementation is here ( and then the python binding will be here )

Here is a revisioned and simplified version of my initial idea

why:

On more complex pipelines (ex multiple stage of NN intervaled with nodes of image manipulation and spatial calculation) keeping the frame in sync can be a real pain with the current interface, the only present way is to put ScriptNodes in between the slow-path and the point of merge and to manually sync the frame again inside the pipeline Also there is no way of knowing which *Config (ex ImageManipConfig) is related to which frame after the generation of it, creating the need to manually marshalling data around to be able to sync a crop region and it's configuration

what:

I'm proposing 2 small change and one addition to the API that is easy to implement and can to enhance a lot the quality of life during development of advanced pipeline while not being a breaking change

1. All type of messages should have the sequenceNumber field (same as ImgFrame) and user configurable (IMHO it should be a base propriety of initial inheritance chain of Buffer itself)
2. All inputs of all node should have a passthrough counterpart on the outputs of the node itself
3. A generic SyncSequenceNumber node that sync messages based on their sequence number

how:

Here pseudo-python of the possible internals of SyncSequenceNumber

input_queue_A   = node.io['inputA']
input_queue_B   = node.io['inputB']

output_queue_A  = node.io['outA']
output_queue_B  = node.io['outB']



def PopA():
    msg = input_queue_A.get()
    return msg.getSequenceNum(), msg
    

def PopB():
    msg = input_queue_B.get()
    return msg.getSequenceNum(), msg


def PopSynced():

    A_sqn, A_msg    = PopA()
    B_sqn, B_msg    = PopB()
    
    while True:

        while   A_sqn < B_sqn:
                    node.info(f"Skipping frame!\t A_sqn: {A_sqn} < B_sqn: {B_sqn}")
                    A_sqn, A_msg    = PopA()


        while   B_sqn < A_sqn:
                    node.info(f"Skipping frame!\t B_sqn: {B_sqn} < A_sqn: {A_sqn}")
                    B_sqn, B_msg    = PopB()


        if      A_sqn == B_sqn:
                    return A_msg, B_msg



while True:

    A_msg, B_sqn = PopSynced()

    output_queue_A.send(A_msg)
    output_queue_B.send(B_sqn)

Then the buffers lengths/pooling are managed/balanced using .setQueueSize(<N>) of the SyncSequenceNumber node inputs, then you connect its outputs to the next node and the setting .setWaitForConfigInput(True) on the receiving node