squared icon indicating copy to clipboard operation
squared copied to clipboard

change: set `ff.env_var_strict_mode` to true

Open aaronsteers opened this issue 2 years ago • 8 comments

@pnadolny13 - We're considering making this the default behavior in a future release. Do you mind testing for us in squared?

This was based on your request to provide a way to set required env vars.

aaronsteers avatar Jul 21 '22 01:07 aaronsteers

@aaronsteers oh nice! I found the issue also https://github.com/meltano/meltano/issues/3348 to refresh my memory. Thanks I'll give it a try.

pnadolny13 avatar Jul 21 '22 13:07 pnadolny13

@aaronsteers I'm not able to get this to work. On main I tried removing this USER_PREFIX to see it fail because its used by dbt-snowflake when I run meltano --log-level=debug invoke dbt-snowflake:test but instead of getting an error related to USER_PREFIX missing I get:

18:49:58  Encountered an error:
Parsing Error
  Env var required but not provided: 'DBT_SNOWFLAKE_USER'

I confirmed that after adding an underscore like this it no longer throws this error because the DBT_SNOWFLAKE_USER value renders as just the underscore which is valid but the nothing about USER_PREFIX gets raised:

      transformers:
      - name: dbt-snowflake
        config:
          user: ${USER_PREFIX}_

Is there something else I should try or could there be a bug? I was expecting it to complain about USER_PREFIX not being set. Any ideas for me?

cc @cjohnhanson

pnadolny13 avatar Jul 21 '22 18:07 pnadolny13

@pnadolny13 This should be resolved now, let me know if you continue to have issues.

cjohnhanson avatar Aug 25 '22 20:08 cjohnhanson

@pnadolny13 - Per @cjohnhanson's comments above, and per the related comment in https://github.com/meltano/squared/issues/354, can you let us know if the latest version of Meltano resolves this, and if so, can we merge in this PR?

aaronsteers avatar Aug 25 '22 22:08 aaronsteers

@aaronsteers @cjohnhanson I'm still not seeing it work on meltano, version 2.5.0 unfortunately. I get a warning Variable '$USER_PREFIX' is missing from the environment. but then still error out on:

15:00:31  Encountered an error:
Parsing Error
  Env var required but not provided: 'DBT_SNOWFLAKE_USER'

Also when I search my debug logs for FF_ references I see an env var MELTANO_FF_STRICT_ENV_VAR_MODE: 'false' but https://docs.meltano.com/reference/settings#ffenv_var_strict_mode has it as MELTANO_FF_ENV_VAR_STRICT_MODE. Are there 2 feature flags? Might be unrelated but seemed weird. I tried setting both of them to true just in case but no luck.

pnadolny13 avatar Aug 26 '22 15:08 pnadolny13

@cjohnhanson - Could you take a look into this ☝️

aaronsteers avatar Aug 26 '22 18:08 aaronsteers

@pnadolny13 -- the docs apparently were not in line with the actual feature flag name. Here is a PR for the docs update.

I'll dig in a little more this morning on why it isn't erroring out when USER_PREFIX not being set

cjohnhanson avatar Aug 29 '22 14:08 cjohnhanson

@pnadolny13 - I've updated the feature flag name here in this PR.

We should retest after

  • https://github.com/meltano/meltano/issues/6696

aaronsteers avatar Aug 31 '22 21:08 aaronsteers