Damon

Implement Dataflow CsvIO example with Custom Template

1

This PR provides two demonstrations. The first is an example processing CSV files into [Schema](https://beam.apache.org/documentation/programming-guide/#what-is-a-schema) aware PCollections using [Apache Beam](https://beam.apache.org/) where we expect different headers. Please consult examples/dataflow-csvio/README for an...

[Task]: Implement TextIO SchemaTransform Provider

1

### What needs to happen? The output of this task is a [TypedSchemaTransformProvider](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/TypedSchemaTransformProvider.java) extension for reading from and writing text files using the TextIO.Read and TextIO.Write transforms, respectively. Briefly described,...

This PR address #21412 with a PubsubSchemaTransformWriteConfiguration implementation. It's design goals are to like-for-like replicate PubsubSchemaIOTransform write configuration details. Subsequent to this PR's approval/merge, the plan is to implement the...

Implement KafkaSchemaTransformReadConfiguration

6

This PR address #21414 with a KafkaSchemaTransformReadConfiguration implementation. It's design goals are to work with a KafkaSchemaTransformReadProvider that extends a [TypedSchemaTransformProvider](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/TypedSchemaTransformProvider.java). Subsequent to this PR's approval/merge, the plan is to...

[FR] Flutter cloud builder

1

## Detailed Feature Information **Which builder is relevant to this request?** None currently listed. **Is your feature request related to a problem? Please describe.** Not applicable. **Describe the solution you'd...

[RRIO] [Throttle] transform that slows down element transmission without an external resource

2

This PR closes #28930 with a PTransform implementation that throttles a PCollection without using any external resources i.e. an external database, queue, etc. Please see #28930 for further details on...

[Prism] Drain behavior

2

### What needs to happen? Under #29669 The following uses an example driven approach via [gherkin syntax](https://cucumber.io/docs/gherkin/reference/) to describe drain behavior. ## Scenario: pipeline reads from Pub/Sub ```gherkin Given pipeline...

# Feature: Throttle a PCollection _As a Beam developer, I want to slow down element throughput of an API request PCollection, so that I can minimize API overusage and save...

This PR submits a plugin for autogenerating terraform modules to launch Dataflow Jobs specific to individual templates. The files listed below are the output of PR #1263. Additionally, some links...

[Feature Request]: Use Secret Manager instead of plain secret input in MQTT to Pub/Sub template

1

### Related Template(s) MQTT to Pub/Sub template ### What feature(s) are you requesting? Replace `password` template parameter with Secret Manager secret name. Currently, the template optionally takes a `password` field...