Hangs in multiprocessing

[devang-kredx]

This test case hangs in Django 2.0 (works without freeze_time)-

from multiprocessing import Pool

from freezegun import freeze_time
from django.test import TestCase


@freeze_time("2018-02-14 11:00:00")
class MultiProcessingTestCase(TestCase):
    """
    Test case to test if pool process of multiprocessing works
    """
    def test_multi_processing(self):
        print("POOLING")
        with Pool(processes=2) as pool:
            pass
        print("DONE")

[spulec]

Closing in favor of https://github.com/spulec/freezegun/issues/202

[ods]

Freezegun still hangs multiprocessing (Python 3.7.0). The problem is in the following chain: Popen.poll(WNOHANG) -> multiprocessing.connection.wait(…, timeout=0). And in wait() there is a loop:

            if timeout is not None:
                deadline = time.time() + timeout

            while True:
                ready = selector.select(timeout)
                if ready:
                    return [key.fileobj for (key, events) in ready]
                else:
                    if timeout is not None:
                        timeout = deadline - time.time()
                        if timeout < 0:
                            return ready

Changing timeout < 0 to timeout <= 0 would solve the problem, but with current code it just never returns.

The following hack applied just before using freezegun solves the problem to me so far:

import multiprocessing.connection
import time

warmed_time = type('module_copy', (), time.__dict__)()
multiprocessing.connection.time = warmed_time

[boxed]

@ods Can you supply a minimal test case? I tried doing the example above but with unittest instead of django.test but that seemed to work fine. Is there something special with the django test runner?

[ods]

Can you supply a minimal test case?

Sure. But my case is not related to django. The following code hangs when executed as is, but works fine without with freeze_time() and with the hack I've described above:

import asyncio
from concurrent.futures import ProcessPoolExecutor

from freezegun import freeze_time

executor = ProcessPoolExecutor()

def sync_func():
    pass

async def main():
    loop = asyncio.get_event_loop()
    with freeze_time():
        await loop.run_in_executor(executor, sync_func)

asyncio.run(main())

[LionelB5]

I'm seeing similar behaviour after upgrading freezegun from 1.0.0 to 1.1.0.

This code snippet reproduces the issue:

from multiprocessing import Pool
# using Pool from `multiprocessing.dummy` also results in a hang 
# from multiprocessing.dummy import Pool

from freezegun import freeze_time


@freeze_time('2020-01-01 00:00:00')
def reproduce_freezegun_hang():
    pool = Pool()
    pool.close()
    pool.join()  # hangs indefinitely here


if __name__ == '__main__':
    reproduce_freezegun_hang()

If I downgrade freezegun from 1.1.0 to 1.0.0 the hang does not happen. Using the workaround mentioned by boxed in this comment solves the issue. Another workaround is to supply tick=True to freeze_time.

I am using Python 3.9.1.

Contents of requirements.txt:

freezegun==1.1.0
    # via -r requirements.in
python-dateutil==2.8.1
    # via freezegun
six==1.16.0
    # via python-dateutil

[AaronDMarasco]

I got it to stop hanging on join() with extend_ignore_list and adding multiprocessing to that. Sadly, that means that since my UUT is in another process from the test bench, applying the tick() doesn't affect the copy that got spun off, so looks like I can't use freezegun for this right now.

I suspect this is also the cause of #359 - unless freezegun implements some kind of multiprocessing manager/namespace to share its state between processes, it's not gonna work. That's what my workaround was, but then I had to add the hooks to my class to explicitly call my time synchronization object, which sucks.

[snnbotchway]

I fixed by setting tick=True like so @freeze_time("2023-06-27 14:30:00", tick=True)