soda-sql icon indicating copy to clipboard operation
soda-sql copied to clipboard

[cloud] Pushing scan metadata to soda cloud fails silently when env vars are not set

Open guillesd opened this issue 4 years ago • 1 comments

Describe the bug When specifying a soda_account in my warehouse.yml file, I would expect that if the environment variables for API_KEY_ID and API_KEY_SECRET are not provided, an exception should be thrown (or at least a warning). However, this is not the case. The only information provided in the logs is the following:

  | Environment variable API_KEY_ID is not set
  | Environment variable API_KEY_SECRET is not set
  | No Soda Cloud account configured

To Reproduce Run soda scan with a warehouse.yml file like the one shown below. If your environment variables are not set (either in the env_vars.yml or by exporting them previous to executing soda, the scan will be successful and no error/warning is thrown.

Context My warehouse.yml file would look something like this:

name: soda_bq
connection:
    type: bigquery
    account_info_json: env_var(BQ_ACCESS)
    auth_scopes:
    - https://www.googleapis.com/auth/bigquery
    dataset: staging
soda_account:
    host: cloud.soda.io
    api_key_id: env_var(API_KEY_ID)
    api_key_secret: env_var(API_KEY_SECRET)

OS: MacOS locally, Ubuntu in prd Python Version: 3.8.11 Soda SQL Version: Latest from master (git+git://github.com/sodadata/soda-sql.git@main#egg=soda-sql-bigquery&subdirectory=packages/bigquery) Warehouse Type: Big Query

guillesd avatar Oct 25 '21 17:10 guillesd

The key seems to be here: https://github.com/sodadata/soda-sql/blob/dc31150e3de38060683d27ed1e5bff5b6ec2802e/core/sodasql/scan/parser.py#L174 I think env_required should be set to true

guillesd avatar Oct 25 '21 17:10 guillesd