appengine-mapreduce icon indicating copy to clipboard operation
appengine-mapreduce copied to clipboard

BigQueryGoogleCloudStorageStoreOutput: fileNamePattern wrong naming.

Open dlazerka opened this issue 8 years ago • 1 comments

BigQueryGoogleCloudStorageStoreOutput requires "fileNamePattern" constructor parameter, but actually it does not accept pattern. At first I wanted to know what pattern should be there (it's undocumented), so I've read the code of SizeSegmentedGoogleCloudStorageFileOutput, which says

   /** @param fileNamePattern a java format string {@link java.util.Formatter} containing one int
   *        argument for the shard number and another int argument for the segment number for e.g.
   *        shard-%04d-segment-%04d.

And supplied a String like "foo-%04d-bar-%04d", but it later throws an exception java.util.MissingFormatArgumentException: Format specifier '04d'.

Debugged, it actually makes its own string: BigQueryFilesToLoad/Job-fileNamePattern/Shard-%04d/file-%04d

I believe constructor parameter should be renamed to something like "jobName".

dlazerka avatar Feb 20 '16 06:02 dlazerka