LAION-AI
Message tree state machine
Message trees are created with an initial prompt. After checking the initial prompt quality via peer-review a tree enters the growing phase in which concurrently prompter/assistant replies and labeling tasks are handed out. When the desired number of messages surpassing the minimum acceptable quality has been collected the tree goes into ranking phase. In ranking phase replies of messages with more than one reply are presented to humans to be ordered from best to worst. When enough ranking results have been collected for all messages the tree enters the ready for scoring phase in which the scoring algorithm combines all user feedback and computes the ranking-scores of all children in the message tree. After the scoring algorithm ran the message tree is completed and enters the ready for export state.
Beside the ready for export additional terminal error-states exist: scoring_failed, aborted_low_grade, halted_by_moderator .. in terminal states no further tasks are handed out to users.
