Instantiate class from config raise error when instantiate class: Checkpoint with config

[viplazylmht]

Describe the bug I'm running GE on Airflow by using GreatExpectationOperator. Things work well until I upgrade GE to v 0.14.6, then I can't instantiate a runtime BaseDataContext with context Config and checkpoint Config.

I debug and found the problem is caused by instantiate_class_from_config __init__(). I moved the hole code back to local Linux environment for testing but the bug still exist.

To Reproduce

1. Instance a runtime data_context via config

from great_expectations.data_context import BaseDataContext
context = BaseDataContext(project_config=data_context_config)

type(context)
# great_expectations.data_context.data_context.BaseDataContext

2. Create checkpoint (the code is nearly same as the part of GE Operator

type(cpoint)
# great_expectations.data_context.types.base.CheckpointConfig

from great_expectations.data_context.util import instantiate_class_from_config
cpj = cpoint.to_json_dict()

checkpoint = instantiate_class_from_config(
          config=cpj,
          runtime_environment={"data_context": context},
          config_defaults={"module_name": "great_expectations.checkpoint"},
      )

# TypeError: Couldn't instantiate class: Checkpoint with config:
# ...
# TypeError: __init__() got an unexpected keyword argument 'site_names'

Yes, I found a outstanding move here: Why we need convert cpoint from CheckpointConfig back to json, then pass to init Checkpoint?

Expected behavior Instantiate successfully Checkpoint via config (expected via CheckpointConfig instead of json).

Temporary fix I removed 4 keys in json_dict that make the error before instantiate_class_from_config:

del cpj['notify_with']
del cpj['slack_webhook']
del cpj['notify_on']
del cpj['site_names']

checkpoint = instantiate_class_from_config( ... )

type(checkpoint)
# great_expectations.checkpoint.checkpoint.Checkpoint

I call it temporary fix because I don't use these configs which make the error now, but maybe I will use it in the future.

Environment:

• Operating System: Linux, Airflow
• Great Expectations Version: 0.14.6

Additional context I use RuntimeDataConnector to connect to a BigQuery data source.

Stack trace:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File ~/.local/lib/python3.8/site-packages/great_expectations/data_context/util.py:115, in instantiate_class_from_config(config, runtime_environment, config_defaults)
    114 try:
--> 115     class_instance = class_(**config_with_defaults)
    116 except TypeError as e:

TypeError: __init__() got an unexpected keyword argument 'site_names'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
Input In [19], in <module>
----> 1 checkpoint = instantiate_class_from_config(
      2           config=cpj,
      3           runtime_environment={"data_context": context},
      4           config_defaults={"module_name": "great_expectations.checkpoint"},
      5       )

File ~/.local/lib/python3.8/site-packages/great_expectations/data_context/util.py:117, in instantiate_class_from_config(config, runtime_environment, config_defaults)
    115     class_instance = class_(**config_with_defaults)
    116 except TypeError as e:
--> 117     raise TypeError(
    118         "Couldn't instantiate class: {} with config: \n\t{}\n \n".format(
    119             class_name, format_dict_for_error_message(config_with_defaults)
    120         )
    121         + str(e)
    122     )
    124 return class_instance

TypeError: Couldn't instantiate class: Checkpoint with config: