modin icon indicating copy to clipboard operation
modin copied to clipboard

TEST: eval_general should assume by default that pandas does not throw an error

Open mvashishtha opened this issue 2 years ago • 3 comments

right now eval_general with default parameters will not raise an assertion if both pandas and modin raise an error, even if that error is due to a mistake in writing the test code. This is dangerous behavior and it has silently swallowed mistakes in test code before. We should have to explicitly state that we are looking for an error.

Right now it's possible to work around this by setting raising_exceptions=(Exception,), but that should be the default.

@modin-project/modin-contributors @modin-project/modin-core @anmyachev I would like your thoughts on this.

mvashishtha avatar Apr 17 '23 15:04 mvashishtha

@mvashishtha I agree, we should change the default behaviour.

anmyachev avatar Apr 17 '23 17:04 anmyachev

+1 on this change

dchigarev avatar Apr 18 '23 15:04 dchigarev

I would add that we may want to check that error messages are "close" (I don't have a good enough definition of "close" here, though).

The case my proposal is aimed at is handling a case when e.g. pandas throws an AttributeError but we've made a typo in test code (and so are also getting AttributeError but not the one we're expecting).

vnlitvinov avatar Jun 21 '23 09:06 vnlitvinov