not_null constraint causing error on severity: warn with warn_if > 0

[scottgigante]

I have a not_null test configured to only warn if the number of failures is >100.

        name: mostly_not_null_column
        tests:
          - not_null:
              config:
                severity: warn
                warn_if: ">100"

It appears this is not triggering dbt_constraints check for failed tests. I get the following error:

on-run-end failed, error:
column "mostly_not_null_column" of relation "my_relation" contains null values