soda-sql
soda-sql copied to clipboard
Make schema/dataset property part of table scan specification
Is your feature request related to a problem? Please describe. Use case: When having multiple datasets or schemas, the mono-configuration of the warehouse.yml expects that all tables are available in one schema/dataset. The idea is that the property should be part of each table YML specification. The below is my attempt to change it to a placeholder schema.
(soda-sql)seba% soda scan warehouse.yml ./tables/code_to_division_mapping.yml
google.api_core.exceptions.NotFound: 404 Not found: Table umg-de:sodasql.code_to_division_mapping was not found in location US
(job ID: xxxxxxxxx)
-----Query Job SQL Follows-----
| . | . | . |
1:SELECT
2: COUNT(*)
3:FROM `sodasql.code_to_division_mapping`
| . | . | . |
| Test error for "row_count > 0": name 'row_count' is not defined
| Executed 1 queries in 0:00:01.690635
| Scan summary ------
| 1 measurements computed
| 1 tests executed
| 1 of 1 tests failed:
| Test test(row_count > 0) failed with measurements null
| Errors occurred!
| [error] Exception during aggregation query
| [test_execution_error] Test "row_count > 0" failed
| Exiting with code 1
| Starting new HTTPS connection (1): collect.soda.io:443
| https://collect.soda.io:443 "POST /v1/traces HTTP/1.1" 200 0
Describe the solution you'd like property schema/dataset should be at table.yml specification and not at warehouse.yml.
Soda SQL Version: 2.1.2
@vijaykiran are you Ok if I propose a PR?
@sebastianvillarroel Sorry for the delay .. can you please write a short summary how it would change the yml ? I'd say this should be "optional" config .. not to cause any issues with existing ymls.