DataflowTemplates
DataflowTemplates copied to clipboard
Support for export specified tables from Cloud Spanner to Cloud Storage Avro template
Currently export dumps entire database. Can we modify this template to include tables argument as well ? Such that only specified tables are exported our from a particular database instance.
This would be a nice addition!
We currently have an internal fork with customizations to exclude tables from a spanner to gcs export (avro format), which takes time to maintain and keep up to date with upstream changes. It would great if the spanner export/import templates supported a UDF to filter tables out of the archive.
Based on recent commits, it looks like this feature has been developed:
- https://github.com/GoogleCloudPlatform/DataflowTemplates/commit/43aab86d5d043703d55dbddf6c57059945f01742
But it looks like it's currently not available for use. Any information on when this would be released?
As mentioned by @peterssonnick, this was already added. The support lagged a little bit due to the release taking a couple of days, but is already available.