great_expectations icon indicating copy to clipboard operation
great_expectations copied to clipboard

`expect_column_max_to_be_between` unexpected exception with `PandasDataset`

Open holaluz-josep-martinez opened this issue 2 years ago • 2 comments

Describe the bug Hi,

I'm running some validations over an in-memory PandasDataset. Two of them are expect_column_min_to_be_between and expect_column_mmax_to_be_between, and they raise a TypeError exception

'>=' not supported between instances of 'Timestamp' and 'str'

even if the column dtype and min_value and max_value are pandas Timestamp (datetime64[ns]) or datetime objects.

I have tried to use the parse_strings_as_datetimes flag; doing so resolves the issue, but triggers the following warning

DeprecationWarning: The parameter "parse_strings_as_datetimes" is deprecated as of v0.13.41 in v0.16. As part of the V3 API transition, we've moved away from input transformation. For more information, please see: https://greatexpectations.io/blog/why_we_dont_do_transformations_for_expectations/

To Reproduce Unzipping and running the following codeshould allow you to reproduce the issue I just described. ge_issues.zip

Environment (please complete the following information):

  • Operating System: Windows 10 Pro, version 21H2
  • Great Expectations Version: 0.15.5

Thank you in advance for your time and help

holaluz-josep-martinez avatar May 19 '22 08:05 holaluz-josep-martinez

Hey @holaluz-josep-martinez ! Thanks for reaching out. We'll review internally and be in touch.

austiezr avatar May 19 '22 16:05 austiezr

Hey @holaluz-josep-martinez ! Thanks for surfacing this; we have eyes on this issue and will be reviewing in the near future.

austiezr avatar Aug 08 '22 17:08 austiezr

Hey @holaluz-josep-martinez ! Are you able to confirm that this is still an issue on the most recent versions of Great Expectations? I'm unable to reproduce (no TypeErrors) from at least great_expectations>=0.15.20

austiezr avatar Nov 07 '22 16:11 austiezr

Hi @austiezr! I'd say it is not an issue in the most recent versions (at least I can't reproduce it either).

holaluz-josep-martinez avatar Nov 08 '22 11:11 holaluz-josep-martinez

Hey @holaluz-josep-martinez ! Great to hear. Closing out for now; feel free to re-open if the issue re-surfaces!

austiezr avatar Nov 08 '22 16:11 austiezr