"Better Processors"

[neovintage]

Feature description

Our plan is for the better processor work for 0.8.0 to include.

• [x] #1284
• [ ] #1285
• [ ] #1286
• [x] #1158
• [x] #1162
• [x] #1156
• [ ] #1287
• [x] #1288
• [ ] #1289
• [ ] #1290
• [ ] #1291
• [ ] #1292

What we currently aren't planning on including in this first pass:

• discovery of processor types (stretch goal)
• open cdc unwrap
• permanent storage
• Splitting and combining records
• Embedded processors

[simonl2002]

This likely won't be in 0.7.0 but the goal is to start the work on this.

[nassor]

Extra features descriptions:

• [ ] Load Processor as plugins as we do with connectors.
• [ ] Have the Processor interface method Close called when Conduit is turning off, so we can release external resources allocated by the Processor
• [ ] Open Telemetry Support: send & return SpanContext to the processor (and connectors as well).
• [ ] Simple Key/Value storage for processors.

[lovromazgon]

Relevant discussion: https://github.com/ConduitIO/conduit/discussions/1141

[raulb]

Documentation will be tackled in this other issue https://github.com/ConduitIO/conduit/issues/1422.

Closing this issue since yesterday "better processors" was shipped as part of 0.9.0.