Investigate WMAgent/T0 required changes to support StepChain with duplicate output module + datatier

[amaltaro]

Impact of the new feature WMAgent

Is your feature request related to a problem? Please describe. PPD has expressed interest in running fork-style StepChain workflows, hence generating the same datatier and output module in different steps of the workflow. This is currently not supported and there is even a protection for this in ReqMgr2.

If we were to allow such workflows to be created in the fork-style configuration, the problem that we would see in the agent is that output from different steps would be consider under the same merge task, hence incorrectly merging unrelated files.

Describe the solution you'd like With this ticket, we need to:

1. investigate what is the current problem with this approach (answer is likely the one above though);
2. which changes would be required to support it (likely affecting how workflow/subscription/fileset is constructed in the relational database - and perhaps in the workload object)
3. which changes on the T0 machinery would be required (likely Tier0Feeder)
4. whether it could be made backwards compatible (dealing either with new and old style workload object construction)

For reference, this ticket addresses this investigation on the ReqMgr2 layer: https://github.com/dmwm/WMCore/issues/11631

Describe alternatives you've considered Alternative is to not support it

Additional context This PR might give some insight on the relevant developments in StepChain: https://github.com/dmwm/WMCore/pull/7998

[vlimant]

Any update on this end? We find ourselves in the same situation that triggered this request in the first place (producing two MINIAODSIM output from the same workflow) again.

I was thinking (naively) that we could "trick it" with https://github.com/cms-sw/cmssw/pull/47501 : i.e. naming the output module differently for each output (one would be MINIAODSIMoutput, the other one MINIAODSIM1output). Would this work ?

[vlimant]

FYI : the trick works just fine, but is very involved / unmaintainable on the submission side