conduit
conduit copied to clipboard
BP: Rewrite built-in processors
- Implement new list of processors defined in the design document
- Make sure each processor is documented and uses the specification generator to extract the specification from Go docs
- Full list:
- [ ]
field.set
- set a field in the record https://github.com/ConduitIO/conduit/pull/1371 - [ ]
field.subset.exclude
- remove fields from the record if they are on the list https://github.com/ConduitIO/conduit/pull/1371 - [ ]
field.subset.include
- remove fields from the record if they are not on the list - [ ]
field.rename
- rename fields https://github.com/ConduitIO/conduit/pull/1371 - [ ]
field.convert
- convert one type into another in a structured payload https://github.com/ConduitIO/conduit/pull/1371 - [ ]
filter
- ack record and remove it from the pipeline https://github.com/ConduitIO/conduit/pull/1371 - [ ]
decode.avro
- decode Avro data into structured data - [ ]
decode.json
- decode a JSON string into structured data - [ ]
encode.avro
- encode structured data into Avro data - [ ]
encode.json
- encode structured data into a JSON string - [ ]
unwrap.kafkaconnect
- unwrap a Kafka Connect payload into the current record - [ ]
unwrap.debezium
- unwrap a Debezium payload into the current record #1393 - [ ]
unwrap.opencdc
- unwrap a OpenCDC payload into the current record #1387 - [ ]
webhook.http
- send a HTTP request for each record #1382 - [ ]
custom.javascript
- execute JavaScript code for each record https://github.com/ConduitIO/conduit/pull/1374
- [ ]
design doc https://github.com/ConduitIO/conduit/blob/main/docs/design-documents/20231117-better-processors.md