DataProfiler icon indicating copy to clipboard operation
DataProfiler copied to clipboard

Bug in popmon_dp_loader_example.ipynb (popmon version incompatibility)

Open ksneab7 opened this issue 2 years ago • 0 comments

General Information: Popmons current version is not compatible with the notebook we have written for integration

Describe the bug:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[6], line 3
      1 pm_data = popmon_dataloader(path, time_index)
----> 3 report_pm_loader = pm_data.pm_stability_report(
      4     time_axis=time_index,
      5     time_width="1w",
      6     time_offset="2015-07-02",
      7     extended_report=False,
      8     pull_rules={"*_pull": [10, 7, -7, -10]},
      9 )
     11 # Save popmon reports
     12 report_pm_loader.to_file(os.path.join(report_output_dir, "popmon_loader_report.html"))

File ~/DataProfiler/venv/lib/python3.8/site-packages/popmon/pipeline/report.py:115, in df_stability_report(df, settings, time_width, time_offset, var_dtype, reference, split, **kwargs)
     91 """Create a data stability monitoring html report for given pandas or spark dataframe.
     92 
     93 :param df: input pandas/spark dataframe to be profiled and monitored over time.
   (...)
    111 :return: dict with results of reporting pipeline
    112 """
    114 if settings is None:
--> 115     settings = Settings(**kwargs)
    117 if len(settings.time_axis) == 0:
    118     settings._set_time_axis_dataframe(df)

File ~/DataProfiler/venv/lib/python3.8/site-packages/pydantic_settings/main.py:71, in BaseSettings.__init__(__pydantic_self__, _case_sensitive, _env_prefix, _env_file, _env_file_encoding, _env_nested_delimiter, _secrets_dir, **values)
     60 def __init__(
     61     __pydantic_self__,
     62     _case_sensitive: bool | None = None,
   (...)
     69 ) -> None:
     70     # Uses something other than `self` the first arg to allow "self" as a settable attribute
---> 71     super().__init__(
     72         **__pydantic_self__._settings_build_values(
     73             values,
     74             _case_sensitive=_case_sensitive,
     75             _env_prefix=_env_prefix,
     76             _env_file=_env_file,
     77             _env_file_encoding=_env_file_encoding,
     78             _env_nested_delimiter=_env_nested_delimiter,
     79             _secrets_dir=_secrets_dir,
     80         )
     81     )

File ~/DataProfiler/venv/lib/python3.8/site-packages/pydantic/main.py:159, in BaseModel.__init__(__pydantic_self__, **data)
    157 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    158 __tracebackhide__ = True
--> 159 __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)

ValidationError: 2 validation errors for Settings
extended_report
  Extra inputs are not permitted [type=extra_forbidden, input_value=False, input_type=bool]
    For further information visit https://errors.pydantic.dev/2.1/v/extra_forbidden
pull_rules
  Extra inputs are not permitted [type=extra_forbidden, input_value={'*_pull': [10, 7, -7, -10]}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.1/v/extra_forbidden

To Reproduce:

  1. pip install popmon
  2. run all cells in the popmon_dp_loader_example.ipynb

Expected behavior: Notebook will error on 6th cell with error described above

ksneab7 avatar Aug 08 '23 13:08 ksneab7