AttributeError: 'NoneType' object has no attribute 'terminate'

[rafaelreuber]

trafficstars

I'm getting the error below when I use to flask.ext.testing.LiveServerTestCase class with python 3.3

Traceback (most recent call last):
  File "C:\Users\Reuber\env_stormy_flask\lib\site-packages\flask_testing\utils.py", line 345, in __call__
    self._spawn_live_server()
  File "C:\Users\Reuber\env_stormy_flask\lib\site-packages\flask_testing\utils.py", line 366, in _spawn_live_server
    self._process.start()
  File "C:\Python33\Lib\multiprocessing\process.py", line 111, in start
    self._popen = Popen(self)
  File "C:\Python33\Lib\multiprocessing\forking.py", line 248, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python33\Lib\multiprocessing\forking.py", line 166, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup _thread.lock failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python33\Lib\runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python33\Lib\runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "C:\Python33\Lib\unittest\__main__.py", line 19, in <module>
    main(module=None)
  File "C:\Python33\Lib\unittest\main.py", line 125, in __init__
    self.runTests()
  File "C:\Python33\Lib\unittest\main.py", line 265, in runTests
    self.result = testRunner.run(self.test)
  File "C:\Python33\Lib\unittest\runner.py", line 168, in run
    test(result)
  File "C:\Python33\Lib\unittest\suite.py", line 67, in __call__
    return self.run(*args, **kwds)
  File "C:\Python33\Lib\unittest\suite.py", line 105, in run
    test(result)
  File "C:\Python33\Lib\unittest\suite.py", line 67, in __call__
    return self.run(*args, **kwds)
  File "C:\Python33\Lib\unittest\suite.py", line 105, in run
    test(result)
  File "C:\Python33\Lib\unittest\suite.py", line 67, in __call__
    return self.run(*args, **kwds)
  File "C:\Python33\Lib\unittest\suite.py", line 105, in run
    test(result)
  File "C:\Users\Reuber\env_stormy_flask\lib\site-packages\flask_testing\utils.py", line 348, in __call__
    self._terminate_live_server()
  File "C:\Users\Reuber\env_stormy_flask\lib\site-packages\flask_testing\utils.py", line 373, in _terminate_live_server
    self._process.terminate()
  File "C:\Python33\Lib\multiprocessing\process.py", line 119, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

[robertlagrant]

Yep - I get the same error. Python 3.4; Windows 10 64bit.

(wenv) C:\rob\github\stargate>nosetests
..Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\dev\Python34\Lib\multiprocessing\spawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "C:\dev\Python34\Lib\multiprocessing\spawn.py", line 116, in _main
    self = pickle.load(from_parent)
EOFError: Ran out of input
E....
======================================================================
ERROR: test_selenium (sg.tests.test_selenium.SeleniumTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\rob\github\stargate\wenv\lib\site-packages\flask_testing\utils.py", line 345, in __call__
    self._spawn_live_server()
  File "c:\rob\github\stargate\wenv\lib\site-packages\flask_testing\utils.py", line 366, in _spawn_live_server
    self._process.start()
  File "C:\dev\Python34\Lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\dev\Python34\Lib\multiprocessing\context.py", line 212, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\dev\Python34\Lib\multiprocessing\context.py", line 313, in _Popen
    return Popen(process_obj)
  File "C:\dev\Python34\Lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\dev\Python34\Lib\multiprocessing\reduction.py", line 59, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class '_thread.lock'>: attribute lookup lock on _thread failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\rob\github\stargate\wenv\lib\site-packages\nose\case.py", line 134, in run
    self.runTest(result)
  File "c:\rob\github\stargate\wenv\lib\site-packages\nose\case.py", line 152, in runTest
    test(result)
  File "c:\rob\github\stargate\wenv\lib\site-packages\flask_testing\utils.py", line 348, in __call__
    self._terminate_live_server()
  File "c:\rob\github\stargate\wenv\lib\site-packages\flask_testing\utils.py", line 373, in _terminate_live_server
    self._process.terminate()
  File "C:\dev\Python34\Lib\multiprocessing\process.py", line 113, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

----------------------------------------------------------------------
Ran 6 tests in 1.637s

[greyli]

Same issue on Windows7 with Python 2.7.

[ProLoDs]

Still an issue with Windos 10 in 2018

[jamiebull1]

Same for Windows 10 in 2019

[RamNewton]

Same for Windows 10 in 2020. Are we missing something here?

[NRGunby]

I've also been getting this (Windows 10, Python 3.7).

I've done some investigation and the root of the problem is that multiprocessing works differently on unix and windows.

On unix, fork() is available, where the OS copies everything for you automatically.

On windows, it is not, and we have to use spawn() instead, which requires the use of pickle to copy variables to the child process. One of the limitations on pickle is that it can copy only copy functions defined at the top level.

The worker that is being spawned is a multiply-nested function ( utils.py -> LiveServerTestCase -> _spawn_live_server() -> worker() -> socket_bind_wrapper() ) so can't be pickled, so we can't start a new thread on windows.

I tried extracting socket_bind_wrapper() and worker() out to the top level of the module, and it fixed this error, but instead of the test working I got the error

AttributeError: Can't pickle local object 'SQLAlchemy.init_app.<locals>.shutdown_session'

I'm not sure if this is just a problem with my application's architecture (I'm not that experienced with flask, this came up while trying to learn it), or if it's intrinsic to flask.

[kennell]

2022, still there

sys

macOS 11.4
Python 3.9.9

versions

Flask==2.0.2
Flask-Testing==0.8.1

[changchiyou]

Got the same error with

macOS Sonoma 14.0
Python 3.10.11
flask 3.0.0
flask-testing 0.8.1

According to https://github.com/jarus/flask-testing/issues/79#issuecomment-696666072 and ChatGPT, here is the workaround for macOS:

https://github.com/jarus/flask-testing/blob/5107691011fa891835c01547e73e991c484fa07f/flask_testing/utils.py#L487

-  self._process = multiprocessing.Process(
+  self._process = multiprocessing.get_context("fork").Process(