public-datasets-pipelines
public-datasets-pipelines copied to clipboard
Support JSON file references in YAML config
Description
To prevent duplicating BigQuery schema definitions in YAML config files (say, when multiple BigQuery tables use the same schema), it might be nice to allow users to create JSON schema files for "shared" schemas and have such files be uploaded to Cloud Composer. These files can then be references for parameters such as schema_object in the GCS to BQ Airflow operator.
Checklist
- [x] I created this issue in accordance with the Code of Conduct.
- [x] This issue is appropriately labeled.