beam icon indicating copy to clipboard operation
beam copied to clipboard

[Bug]: --dataflow_job_file does not work with direct runner

Open dannikay opened this issue 1 year ago • 0 comments

What happened?

I'm trying to inspect the workflow graph generated by Beam without submitting the job to Dataflow. So I used the following command:

python -m wordcount --runner DirectRunner --dataflow_job_file ./debug.txt

However the "dataflow job file" was not dumped and there is no error executing the above command:

INFO:apache_beam.runners.worker.statecache:Creating state cache with size 104857600
WARNING:apache_beam.io.filebasedsink:Deleting 1 existing files in target path matching: -*-of-%(num_shards)05d
INFO:apache_beam.io.filebasedsink:Starting finalize_write threads with num_shards: 1 (skipped: 0), batches: 1, num_threads: 1
INFO:apache_beam.io.filebasedsink:Renamed 1 shards in 0.00 seconds.

Issue Priority

Priority: 3 (minor)

Issue Components

  • [X] Component: Python SDK
  • [ ] Component: Java SDK
  • [ ] Component: Go SDK
  • [ ] Component: Typescript SDK
  • [ ] Component: IO connector
  • [ ] Component: Beam YAML
  • [ ] Component: Beam examples
  • [ ] Component: Beam playground
  • [ ] Component: Beam katas
  • [ ] Component: Website
  • [ ] Component: Spark Runner
  • [ ] Component: Flink Runner
  • [ ] Component: Samza Runner
  • [ ] Component: Twister2 Runner
  • [ ] Component: Hazelcast Jet Runner
  • [ ] Component: Google Cloud Dataflow Runner

dannikay avatar Jun 19 '24 19:06 dannikay