beam
beam copied to clipboard
[Feature Request]: Support int32 for properties of Cross lang transformations written in Java
What would you like to happen?
All cross lang transformations that are implemented in java that have a python wrapper that use the old ExternalTransform API such as KafkaIO are forced to have int64 properties since the translation layer converts a python int to int64. So if an int32 property is added to the java transform, unless the python wrapper specifies an np.int32 (which is clunky since it needlessly adds an np dependency), the python pipeline will error.
Issue comes from this file: https://github.com/apache/beam/blob/0f2e1963987f1fbb3329016d8c862639ed4fbe43/sdks/python/apache_beam/typehints/schemas.py#L4
Issue Priority
Priority: 3 (nice-to-have improvement)
Issue Components
- [X] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner