Bizarre errors from Pandas when using the decorator

[anentropic]

Wanted to use this but I get bizarre errors from Pandas when I add the decorator to my test function.

The test and the code being tested do not execute any Pandas methods directly, but one of my project modules imports pandas - the error is occurring at import time.

  File "/Users/anentropic/Documents/Dev/Work/anentropic/backend/stats/users.py", line 6, in <module>
    import pandas as pd
  File "/Users/anentropic/.virtualenvs/anentropic/lib/python2.7/site-packages/pandas/__init__.py", line 7, in <module>
    from . import hashtable, tslib, lib
  File "pandas/tslib.pyx", line 614, in init pandas.tslib (pandas/tslib.c:81486)
  File "pandas/tslib.pyx", line 573, in pandas.tslib.NaTType.__new__ (pandas/tslib.c:12539)
TypeError: __new__() takes exactly one argument (4 given)

[spulec]

Which version of freezegun are you on? We've had this issue in the past, but I thought it was fixed with https://github.com/spulec/freezegun/issues/13

[anentropic]

Installed current pip version today On 26 Jun 2015 19:30, "Steve Pulec" [email protected] wrote:

Which version of freezegun are you on? We've had this issue in the past, but I thought it was fixed with #13 https://github.com/spulec/freezegun/issues/13

— Reply to this email directly or view it on GitHub https://github.com/spulec/freezegun/issues/98#issuecomment-115825695.

[anentropic]

Ah, I have read the other issue and I see it is the same symptom but a different case

It was fixed in #13 for the case of import freezegun kills non-frozen tests which import Pandas

My case is a frozen test that contains code that imports Pandas

Is there any insight into why Pandas spits the dummy? Is it something that could be solved?

I'm currently using python-libfaketime instead

[spulec]

I think Pandas does some C-level manipulation or use of the datetime/time modules. I've dug into it a bit, but not found a way that we could work around it.

[vitawasalreadytaken]

Just ran into this problem too – my frozen test code was importing Pandas, and Pandas import failed with

TypeError: type 'pandas.tslib._Timestamp' is not dynamically allocated but its base type 'FakeDatetime' is dynamically allocated

I got it to work by explicitly importing Pandas before the test code runs.

[spulec]

I'm going to close this. Please reopen if you confirm it is still an issue

[bhrutledge]

FYI, this still seems to be an issue, even using the latest versions of Pandas and freezegun:

@freeze_time("2020-01-01")
def test_freeze_time_pandas():
    import pandas

Python 2:

___________________________ test_freeze_time_pandas ____________________________
test_model_factory.py:89: in test_freeze_time_pandas
    import pandas
../../../../../venv/lib/python2.7/site-packages/pandas/__init__.py:25: in <module>
    from pandas import hashtable, tslib, lib
pandas/tslib.pyx:918: in init pandas.tslib (pandas/tslib.c:119031)
    ???
pandas/tslib.pyx:805: in pandas.tslib.NaTType.__new__ (pandas/tslib.c:16573)
    ???
E   TypeError: __new__() takes exactly one argument (4 given)

Python 3:

___________________________ test_freeze_time_pandas ____________________________
test_model_factory.py:89: in test_freeze_time_pandas
    import pandas
../../../.tox/py36/lib/python3.6/site-packages/pandas/__init__.py:25: in <module>
    from pandas import hashtable, tslib, lib
pandas/tslib.pyx:1051: in init pandas.tslib (pandas/tslib.c:116027)
    ???
E   TypeError: type 'pandas.tslib._Timestamp' is not dynamically allocated but its base type 'FakeDatetime' is dynamically allocated

This is admittedly a contrived example, and in my case it seems like I can avoid this by using freeze_time as a context manager, or reworking the complex import chain. However, I can imagine a situation where one or both of those things might not be possible/practical.

[bhrutledge]

To give a concrete example of this suggestion from @ze-phyr-us:

I got it to work by explicitly importing Pandas before the test code runs.

I was able to do this by adding import pandas to my global conftest.py.

[marcinmilewski1]

Why this issue is closed? It's still a problem.

Could you please reopen @spulec ?

[sshishov]

Apparently the issue can be fixed using workaround by importing pandas somewhere before the test, like in conftest. But anyway it looks like a workaround (especially if you need to add some ignore linters - unused import etc). I hope that some time in future we would be able to figure out the problem and fix it...

[hedleyroos]

Importing pandas early is not always a good idea, since it consumes a lot of memory without actually doing anything. If you have a project that can be invoked in many different ways (think of one code base that can run both as a webserver and as a task queue processor in Docker containers) then you want to avoid that pandas import if you're never going to need it. You don't always have the option of splitting the code base.

My workaround is to have a conditional early import when running tests.