beam

beam copied to clipboard

Python Direct Runner doesn't support both streaming & non streaming sources

4

Please see Stack Overflow discussion: [https://stackoverflow.com/questions/68125864/transform-node-appliedptransform-was-not-replaced-as-expected-error-with-the-dir](https://stackoverflow.com/questions/68125864/transform-node-appliedptransform-was-not-replaced-as-expected-error-with-the-dir) When I create a GCS source & a Pub Source and try to flatten both, there is an error because of some incompatible transformation...

damccorm

[Bug]: Reject illformed GBK Coders

3

### What happened? ``` p.apply( Create.of(KV.of("X", "Y")) .withCoder(KvCoder.of(StringUtf8Coder.of(), StringUtf8Coder.of()))) .apply(GroupByKey.create()) .setCoder( SerializableCoder.of( TypeDescriptors.kvs( TypeDescriptors.strings(), TypeDescriptors.iterables(TypeDescriptors.strings())))); ``` The pipeline above exemplifies how it is, at the moment, acceptable to author pipelines...

pskevin

Add java Bigquery IO known issue to beam 2.40 release blogpost

1

------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R:...

johnjcasey

- update commits link on contact-us page from `https://s.apache.org/beam-slack-channel` to `https://lists.apache.org/[email protected]` ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: -...

bullet03

Adds Testcontainers with SSL config for SSL-specific testing

3

Some features of the ElasticsearchIO connector[1] can only be verified by testing against a cluster with SSL and user/password authentication enabled.  We should provide a way to test that kind...

damccorm

### What happened? Hello, I was trying to use the WriteToKafka from apache_beam.io.kafka. Evrything in pipeline is working but when I added the WriteToKafka I have this error: ` raise...

imadmoussa1

Introduce proper retry handling for Python BigTable I/O

1

Currently we only have a sink and the implementation is here: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigtableio.py For example, we should properly identify various errors codes and retry failed mutations appropriately. Imported from Jira [BEAM-13849](https://issues.apache.org/jira/browse/BEAM-13849)....

damccorm

[Playground] [Task]: Support for a default category of an example

1

### What needs to happen? Currently an example can have multiple categories. In the frontend, it gets highlighted in all of them, and they all are initially expanded:  This...

alexeyinkin

[Tour of Beam] [Task] Learning content for "Core Transforms" module

3

### What needs to happen? Create learning content and challenge for Tour of Beam::Core Transforms module consisting of following units: - Core Transforms/Map/Map Elements - Core Transforms/Map/FlatMapElements - Core Transforms/Map/ParDo...

olehborysevych

[Feature request] [Tour of Beam] CI/CD for project - Platform environment in GCP

1

### What needs to happen? CI/CD for project - Platform environment in GCP ### Issue Priority Priority: 3

olehborysevych