connect
connect copied to clipboard
Fancy stream processing made operationally mundane
Ensure that the Redis processor has the args_mapping field. Fixes #1371. It now outputs the following: ``` ERRO Config lint error @service=benthos lint="./tmp/config_redis_cache.yaml: line 7: field args_mapping is required" ERRO...
Hi, I am using benthos 4.5.0 and i am using gcp_cloud_storage input and gcp_bigquery as an output (streaming mode). I have a folder containing a bunch of json logs and...
Manually tested and it looks pretty good. If the schema is: ``` { "type": "record", "name": "heartbeat_v3", "namespace": "outbox_events", "fields": [ { "name": "heartbeat", "type": [ "null", "int" ], "default":...
Hello! We use benthos in streaming mode and processing several data streams. Is there a way to set a variable (or something like that) that we can use to configure...
consider the below config: ``` processors: - log: level: DEBUG message: '${! content() + meta() }' ``` The above will fail; meta().string() should be used instead. Expected: benthos should log...
Currently we have to run Benthos as a Lambda to consume from a Dynamo DB stream. It would be nice to consume this as a stream in a long running...
**Problem** Missing `args_mapping` from redis processor causes panic during runtime. **Example** ```yaml input: stdin: {} pipeline: processors: - redis: url: redis://localhost:6379 command: foo output: stdout: {} ``` **Result** ```shell ❯...
This change adds a top-level `shutdown_delay` config option that sets an optional shutdown delay duration. This option can be used to delay the start of the shutdown procedure. This is...
This parameter allows to enable long-polling and reduce costs.
This is something we are actively using. JSONPath is very useful when handling unstructured/schemaless data.