beam icon indicating copy to clipboard operation
beam copied to clipboard

[Bug]: Better error message for specifying inputs on chain pipeline in Beam YAML

Open Polber opened this issue 1 year ago • 0 comments

What happened?

When specifying explicit inputs to transforms in Beam YAML, the error message is difficult to decipher.

For example, running the following pipeline:

pipeline:
  type: chain
  transforms:
    - type: Create
      config:
        elements:
          - id: 1
    - type: LogForTesting
      input: Create

will result in the following error message:

jsonschema.exceptions.ValidationError: 'Create' should not be valid under {} around line 475

Failed validating 'not' in schema['properties']['pipeline']['allOf'][2]['then']['properties']['transforms']['items']['properties']['input']:
    {'not': {}}

On instance['pipeline']['transforms'][1]['input']:
    'Create'

This should instead print something like "Cannot define explicit inputs on chain pipelines"

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • [ ] Component: Python SDK
  • [ ] Component: Java SDK
  • [ ] Component: Go SDK
  • [ ] Component: Typescript SDK
  • [ ] Component: IO connector
  • [X] Component: Beam YAML
  • [ ] Component: Beam examples
  • [ ] Component: Beam playground
  • [ ] Component: Beam katas
  • [ ] Component: Website
  • [ ] Component: Infrastructure
  • [ ] Component: Spark Runner
  • [ ] Component: Flink Runner
  • [ ] Component: Samza Runner
  • [ ] Component: Twister2 Runner
  • [ ] Component: Hazelcast Jet Runner
  • [ ] Component: Google Cloud Dataflow Runner

Polber avatar Aug 27 '24 18:08 Polber