great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

Misleading exception catching

Open akerone opened this issue 2 years ago • 2 comments

Describe the bug In great_expectations/data_context/store/store_backend.py, at least from versions 0.14.0 to 0.14.8, line 103.

This exception tells us that there is something wrong in great_expectations.yaml, when in reality it could be anything. In our case, it was a missing requirement to the google-cloud-storage library from pip. Without a proper stack trace, it took us too much time to trace down.

To Reproduce Steps to reproduce the behavior:

  1. Create a new venv
  2. Install great expectations
  3. Configure a Store to read from GCS (with google-cloud-storage not installed)
  4. Try following the notebook after great_expectations datasource new

Expected behavior Printing the full stack trace of the error that occurred, rather than just a simple warning text.

Environment (please complete the following information):

  • Operating System: Windows 10
  • Great Expectations Version: 0.14.0 to 0.14.8 (ad minimum)

Additional context image (1)

akerone avatar Mar 01 '22 15:03 akerone

Thanks for raising this, @akerone - we will review and be in touch!

talagluck avatar Mar 02 '22 19:03 talagluck

Hey @akerone ! We have eyes on this issue and will be reviewing in the near future.

austiezr avatar Aug 08 '22 17:08 austiezr

Hey @akerone is this still an issue in the latest version of GX?

rdodev avatar Apr 06 '23 15:04 rdodev

Hi,

I don't know since we stopped using Great Expectations altogether due to this and other bugs.

akerone avatar Apr 06 '23 15:04 akerone

Hey @akerone This is truly disappointing to hear :( If it's not too much to ask, would you mind sharing if you're using a different DQ tool or did you device a homegrown solution?

rdodev avatar Apr 06 '23 16:04 rdodev

Since we decided to use dbt for our ELT processes, we're using dbt tests w/ the dbt-expectations package which tries to replicate most GX tests in "dbt-native" macros. No need for two separate systems for ELT and testing.

akerone avatar May 11 '23 08:05 akerone