gcp-ingestion
gcp-ingestion copied to clipboard
File Beam PR for allowing unbounded running without numShards
It appears that the PR resolving https://issues.apache.org/jira/browse/BEAM-1438 was intended to allow unbounded jobs to write files without specifying a number of shards, but there is still an argument check that throws an error on compile for this situation.
It may be that running without specifying a number of shards is still considered too dangerous in the general case, but we should see if this can be relaxed.