Wizard Step: Completion Step

[grotlue]

• When switching to this step, the input and everything else configured in the previous steps will be excuted
• It shows a list of all the steps that have been completed
• It shows errors and error messages of failed steps

[tellistone]

Maybe we present the text as showing what will be done (eg. Input "my Big Input" will be started!) rather than what is already done

We discussed performing the actions sequentially, and showing user error of which step fails in event of fail

we discussed rolling anything we created here back, in the event of a fail

[tellistone]

if the input no longer exists or is not in setup mode when you press OK, handle that exception

[grotlue]

Steps that are executed

Always

• Start Input - put /cluster/inputstates/{inputId}

Illuminate

• Not in scope for this cycle, but with adding the Illuminate flow, we will have more steps here (will be added in a seperate issue)

Routing

Option 1 - Use default stream

• No extra routing necessary

Option 2 - Use existing stream

• Create Pipeline Rule - post /system/pipelines/rule
• Add Pipeline Rule to Default Pipeline (new) Stage 0

Option 3 - New Stream

• Create Stream - post /streams
• Create empty Pipeline named after the stream (if option toggled on) - post /system/pipelines/pipeline
  • Connect new Pipeline to Stream - post /system/pipelines/connections/to_stream
• Add Pipeline Rule to Default Pipeline (new) Stage 0

[patrickmann]

Regarding the APIs for routing: For options 1 and 2, I suggest we introduce a new API put /system/pipelines/pipeline. Same as put /system/pipelines/pipeline/{id}, but BE will add the rule to default pipeline (and create one, if it doesn't yet exist). Since the payload source field includes the pipeline name, we need to use the same in FE and BE.

Alternatively, the new endpoint could just take an input ID as the parameter. BE would then create the rule and add it to a default pipeline. This would reduce coupling between FE and BE.

Re option 1: What default stream is this referring to? According to #20568, user either selects an existing stream or creates a new one.

[grotlue]

Re option 1: What default stream is this referring to? According to https://github.com/Graylog2/graylog2-server/issues/20568, user either selects an existing stream or creates a new one.

Default is: All messages (Default Stream)

Both API options sound fine for me. Both would need the pipeline name (nullable -> which means default) and the input id passed, right?

[grotlue]

Discussion outcome 19.11.2024

• For messages to work with Illuminate, we need them to be in the Default Stream first. That's one of the reasons why we prefer pipeline rules over stream rules.
• We need an immutable pipeline for wizard-related routing (e.g. "built-in" or "system" pipeline)
  • we always add rule to stage 0
  • in this case immutable means:
    • not deletable
    • can't be renamed
    • connections can't be changed
    • rules can be deleted, added, edited
  • Alternative approach: pipeline is deletable, but will be recreated immediately
    • That would lead to users potentially messing up routing that was created by a wizard (we can live with that though)
    • less effort as it won't need UI adjustments
    • we decided to go for this option even though the pipeline won't be immutable in this case