phoebe-project
Segmentation fault with new dependencies (python/numpy/astropy)
Discussed in https://github.com/phoebe-project/phoebe2/discussions/778
Originally posted by rj627 August 26, 2023
Hi -- I was having no issues running Phoebe until I upgraded my Python version to 3.11.4 a few days ago. I'm trying to run the inverse examples notebook locally, and it gives a segmentation fault during the run_compute() line.
I guess I'm not using the ellc compute that is suggested in the notebook (because I can't get ellc to install properly on my M1 Mac), but I was having no issues with run_compute() before I upgraded. Any help would be appreciated!
I've also been having issues with plotting, it seems to be complaining about a "MaskedNDArray," but that seems like an astropy issue and not a phoebe issue. Thanks again!
Python packages installed:
python: 3.11.4
phoebe: 2.4.11
numpy: 1.25.2
matplotlib: 3.7.2
astropy: 5.3.2
Laptop: Late 2020 M1 MacBook Pro
Thanks! I created a new environment with python=3.9.15 (as that's what I use for some of my other environments) and am having issues installing phoebe via pip. The gevent module seems to be the culprit, but not sure how to get around this. I tried installing with sudo as well, but that didn't seem to work either.
running build_ext
generating cffi module 'build/temp.macosx-11.1-arm64-cpython-39/gevent.libuv._corecffi.c'
creating build/temp.macosx-11.1-arm64-cpython-39
Running '(cd "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/deps/libev" && sh ./configure -C > configure-output.txt )' in /private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f
configure: error: in `/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/deps/libev':
configure: error: C compiler cannot create executables
See `config.log' for more details
Traceback (most recent call last):
File "/Users/rahuljayaraman/miniforge3/envs/phoebe_env/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/Users/rahuljayaraman/miniforge3/envs/phoebe_env/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/rahuljayaraman/miniforge3/envs/phoebe_env/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 434, in build_wheel
return self._build_with_temp_dir(
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 419, in _build_with_temp_dir
self.run_setup()
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 341, in run_setup
exec(code, locals())
File "<string>", line 468, in <module>
File "<string>", line 337, in run_setup
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/normal/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 364, in run
self.run_command("build")
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1233, in run_command
super().run_command(command)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/cffi/setuptools_ext.py", line 143, in run
ext.sources[0] = make_mod(self.build_temp, pre_run)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-build-env-fht3c1ew/overlay/lib/python3.9/site-packages/cffi/setuptools_ext.py", line 128, in make_mod
pre_run(ext, ffi)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/_setuputils.py", line 381, in pre_run
action()
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/_setuplibev.py", line 55, in configure_libev
system(libev_configure_command)
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/_setuputils.py", line 195, in system
if _system(cmd, cwd=cwd, env=env, **kwargs):
File "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/_setuputils.py", line 191, in _system
return check_call(cmd, cwd=cwd, env=env, **kwargs)
File "/Users/rahuljayaraman/miniforge3/envs/phoebe_env/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '(cd "/private/var/folders/ft/3npvnj7106x8rv41vmyd1z780000gn/T/pip-install-s6j4dxd7/gevent_7f0603b35e7a48ac99c213938861f79f/deps/libev" && sh ./configure -C > configure-output.txt )' returned non-zero exit status 77.
Can you install gevent directly (pip install gevent)? If not, then that might be a problem on their end, or you can try to specifically request an older version of gevent before installing phoebe.
I managed to get gevent to work and installed phoebe in an environment with Python version 3.9.0, and downloaded astropy==5.1, numpy==1.22.3. I also reverted scipy to v1.9, but still the segmentation fault occurs when run_compute() is called.
@horvatm - any other logs or information that would be helpful to track this down (I'm guessing the segfault is likely coming from our C-code if its within run_compute, but I guess it could also be a dependency)?
Are any calls to run_compute() segfaulting or just that particular one? I just tried to reproduce but it's working fine on my end.
Thanks for suggesting this -- I ran some old code that calls run_compute and it worked fine! So it seems like it's a problem with this particular notebook on my machine.
I dug a bit deeper into the aforementioned notebook and found that commenting out the line where the ecc value is set to 0.2 avoids a segfault, but keeping that line creates one.
I experimented with various ecc values, but found that the only one that avoids a segfault is 0.01. Anything > 0.02 creates a segfault. Why this could be the case, I don't know...
Hi. In phoebe we don't build
gevent.libuv._corecffi.c
as far as I know. This is not part of libphoebe.
Thanks for suggesting this -- I ran some old code that calls
run_computeand it worked fine! So it seems like it's a problem with this particular notebook on my machine.
When I go all the way down the notebook and do the line b.run_compute(irrad_method='none', model='after_estimators', overwrite=True), it segfaults.
Could you possibly get a traceback to see what segfaults exactly? Running the script through gdb could shed light into what's going on.
Sure, here is a full bt when including the eccentricity value --
1% 14/1105 [00:00<00:29, 37.41it/s]Process 55711 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x600020000050)
frame #0: 0x000000014e583a3c libphoebe.cpython-39-darwin.so`Tmarching<double, Tmisaligned_roche<double> >::check_bad_pairs(std::__1::vector<Tmarching<double, Tmisaligned_roche<double> >::Tvertex, std::__1::allocator<Tmarching<double, Tmisaligned_roche<double> >::Tvertex> >&, double const&) + 112
libphoebe.cpython-39-darwin.so`Tmarching<double, Tmisaligned_roche<double> >::check_bad_pairs:
-> 0x14e583a3c <+112>: ldp d5, d6, [x2, #0x60]
0x14e583a40 <+116>: fmul d6, d3, d6
0x14e583a44 <+120>: fmadd d5, d2, d5, d6
0x14e583a48 <+124>: ldr d6, [x2, #0x70]
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x600020000050)
* frame #0: 0x000000014e583a3c libphoebe.cpython-39-darwin.so`Tmarching<double, Tmisaligned_roche<double> >::check_bad_pairs(std::__1::vector<Tmarching<double, Tmisaligned_roche<double> >::Tvertex, std::__1::allocator<Tmarching<double, Tmisaligned_roche<double> >::Tvertex> >&, double const&) + 112
frame #1: 0x000000014e5934bc libphoebe.cpython-39-darwin.so`Tmarching<double, Tmisaligned_roche<double> >::triangulize_full_clever(double*, double*, double const&, unsigned int const&, std::__1::vector<T3Dpoint<double>, std::__1::allocator<T3Dpoint<double> > >&, std::__1::vector<T3Dpoint<double>, std::__1::allocator<T3Dpoint<double> > >&, std::__1::vector<T3Dpoint<int>, std::__1::allocator<T3Dpoint<int> > >&, std::__1::vector<double, std::__1::allocator<double> >*, double const&) + 6172
frame #2: 0x000000014e55fe18 libphoebe.cpython-39-darwin.so`roche_misaligned_marching_mesh(_object*, _object*, _object*) + 3112
frame #3: 0x00000001000b3968 python`cfunction_call + 80
frame #4: 0x0000000100060e28 python`_PyObject_Call + 156
frame #5: 0x0000000100165710 python`_PyEval_EvalFrameDefault + 30088
frame #6: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #7: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #8: 0x0000000100063ed4 python`method_vectorcall + 164
frame #9: 0x0000000100168cac python`call_function + 572
frame #10: 0x00000001001654d8 python`_PyEval_EvalFrameDefault + 29520
frame #11: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #12: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #13: 0x0000000100063ed4 python`method_vectorcall + 164
frame #14: 0x0000000100168cac python`call_function + 572
frame #15: 0x00000001001654d8 python`_PyEval_EvalFrameDefault + 29520
frame #16: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #17: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #18: 0x0000000100063ed4 python`method_vectorcall + 164
frame #19: 0x0000000100168cac python`call_function + 572
frame #20: 0x00000001001654d8 python`_PyEval_EvalFrameDefault + 29520
frame #21: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #22: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #23: 0x0000000100063ed4 python`method_vectorcall + 164
frame #24: 0x0000000100060c50 python`PyVectorcall_Call + 156
frame #25: 0x0000000100165710 python`_PyEval_EvalFrameDefault + 30088
frame #26: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #27: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #28: 0x0000000100063ed4 python`method_vectorcall + 164
frame #29: 0x0000000100060c50 python`PyVectorcall_Call + 156
frame #30: 0x0000000100165710 python`_PyEval_EvalFrameDefault + 30088
frame #31: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #32: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #33: 0x0000000100063ed4 python`method_vectorcall + 164
frame #34: 0x0000000100060c50 python`PyVectorcall_Call + 156
frame #35: 0x0000000100165710 python`_PyEval_EvalFrameDefault + 30088
frame #36: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #37: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #38: 0x0000000100060c50 python`PyVectorcall_Call + 156
frame #39: 0x0000000100165710 python`_PyEval_EvalFrameDefault + 30088
frame #40: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #41: 0x0000000100061058 python`_PyFunction_Vectorcall + 240
frame #42: 0x0000000100063ed4 python`method_vectorcall + 164
frame #43: 0x0000000100168cac python`call_function + 572
frame #44: 0x00000001001654d8 python`_PyEval_EvalFrameDefault + 29520
frame #45: 0x000000010015db40 python`_PyEval_EvalCode + 2968
frame #46: 0x00000001001c11a4 python`PyRun_FileExFlags + 464
frame #47: 0x00000001001c0480 python`PyRun_SimpleFileExFlags + 436
frame #48: 0x00000001001e328c python`Py_RunMain + 2916
frame #49: 0x00000001001e4400 python`pymain_main + 1256
frame #50: 0x0000000100005de0 python`main + 56
frame #51: 0x0000000197e9ff28 dyld`start + 2236
Am I reading this correctly, the bad access is happening in
check_bad_pairs
Yes, there is a possibility that something goes wrong there. It would be nice to have an example that breaks on any machine.
Well, I ran run_compute() for an example for my project and it segfaults on the same function (misaligned_roche), but at the call to triangulize_full_clever. I'm happy to attach my Python script if it would help with debugging.
Here's what I have; the last line (call to run_compute()) is where the seg fault occurs.
(GitHub does not support .py files, so I've uploaded it as a .txt file, but all the code should work.)
I just tried running this code on my end and still cannot reproduce a segfault. Here is the output I'm getting:
(release) andrej@albireo:~/projects/phoebe/tests/segfault$ python segfault.py
/home/andrej/projects/phoebe/venvs/release/lib/python3.10/site-packages/astropy/units/quantity.py:611: RuntimeWarning: invalid value encountered in sqrt
result = super().__array_ufunc__(function, method, *arrays, **kwargs)
100%|██████████████████████████████████████| 1518/1518 [00:11<00:00, 130.17it/s]
['comments', 'use_server', 'lc_datasets', 'phase_bin', 'phase_nbins', 'orbit', 'analytical_model', 'interactive', 't0_near_times', 'expose_model']
['lc01']
100%|███████████████████████████████████████| 1518/1518 [01:27<00:00, 17.33it/s]
I will try it on a mac in the next few days (got to grab a mac from somewhere) and see if I can reproduce it.
I think this is certainly a Mac issue, as I had no problem running this code on one of the remote clusters. For now, I'll probably just run code there as a temporary solution, but would be good to figure out why it's erroring out on my Mac...
I think this may now be fixed by #884 included in the 2.4.14 release, so I'll close this issue since its quite old, but please feel free to re-open or create a new issue if you still run into any problems.