phoebe2 icon indicating copy to clipboard operation
phoebe2 copied to clipboard
verified publisher icon phoebe-project

liphoebe results in "undefined symbol" error

Open westobin opened this issue 2 years ago • 5 comments

Hello,

I just upgraded to 2.4.11 on my local and cluster. I have no problems with the local installation, but the cluster installation is giving me an odd error:

base) westobin@login2:~> python3 -c "import phoebe; print(phoebe.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "~/miniconda3/lib/python3.9/site-packages/phoebe/__init__.py", line 377, in <module>
    from .atmospheres.passbands import install_passband, uninstall_passband, uninstall_all_passbands, download_passband, list_passband_online_history, update_passband_available, update_passband, update_all_passbands, list_all_update_passbands_available, list_online_passbands, list_installed_passbands, list_passbands, list_passband_directories, get_passband
  File "~/miniconda3/lib/python3.9/site-packages/phoebe/atmospheres/passbands.py", line 22, in <module>
    import libphoebe
ImportError: ~/miniconda3/lib/python3.9/site-packages/libphoebe.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv

I assume that there's a conflict with something else that is loaded on the system, but I do not know what.

Prior installation of 2.4.2 was working without any errors at the time of the upgrade.

Upgrade process to 2.4.11 had no installation warnings or errors. At first, I did not upgrade any of the dependencies, however, after receiving this error I upgraded the dependencies and then reinstalled phoebe (again with no warnings or errors). It seems that it always throws this error on importing phoebe.

I then downgraded to Phoebe 2.4.10, which successfully completed the version check operation:

(base) westobin@login2:~> python3 -c "import phoebe; print(phoebe.__version__)"
2.4.10

Any ideas on what I might try to resolve this issue for version 2.4.11?

Thanks!

  • Wes

westobin avatar Sep 11 '23 13:09 westobin

Hi Wes - there weren't any changes to the C-code (which is what is throwing the error) between 2.4.10 and 2.4.11. Are there any differences in your compilers or dependencies (numpy, scipy, astropy, etc) between your working 2.4.10 and failing 2.4.11 installation environments?

kecnry avatar Sep 11 '23 13:09 kecnry

No changes in the dependencies. I simply ran: Python3 -m pip install phoebe==2.4.10 And it installed 2.4.10 and uninstalled 2.4.11 with no changes to the dependencies at all. I was using the same compiler in all cases...

westobin avatar Sep 11 '23 17:09 westobin

@westobin, is this still an issue with 2.4.12? Can you tell us which version of c++ and python you have installed on your cluster?

aprsa avatar Jan 21 '24 00:01 aprsa

Hi.

If the compiling was successful, then the "missing symbols" while running means that dynamical libraries on which libphoebe depends are not found. In the case of message

undefined symbol: _ZSt28__throw_bad_array_new_lengthv

indicates, that it does not find libstdc++ with which the compilation was performed.

horvatm avatar Jan 21 '24 10:01 horvatm

Yes, I am still receiving the error. Note that this is only happening on the HPC Cluster and not on my local machine. I have tried installations using different gcc compiler modules (11.1.0, 12.2.0) with no luck. However, Phoebe 2.4.10 still installs and works fine.

Phoebe 2.4.11+ installs and works fine on my local machine.

Here's the full output from upgrading to 2.4.12 through trying to import phoebe today:

(base) westobin@login2:~> python3 -m pip install phoebe -U
Requirement already satisfied: phoebe in ./miniconda3/lib/python3.9/site-packages (2.4.10)
Collecting phoebe
  Downloading phoebe-2.4.12.tar.gz (77.8 MB)
     |████████████████████████████████| 77.8 MB 6.7 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: scipy in ./miniconda3/lib/python3.9/site-packages (from phoebe) (1.11.2)
Requirement already satisfied: tqdm in ./miniconda3/lib/python3.9/site-packages (from phoebe) (4.66.1)
Requirement already satisfied: astropy in ./miniconda3/lib/python3.9/site-packages (from phoebe) (5.3.3)
Requirement already satisfied: flask-socketio in ./miniconda3/lib/python3.9/site-packages (from phoebe) (4.3.2)
Requirement already satisfied: gevent-websocket in ./miniconda3/lib/python3.9/site-packages (from phoebe) (0.10.1)
Requirement already satisfied: flask-cors in ./miniconda3/lib/python3.9/site-packages (from phoebe) (4.0.0)
Requirement already satisfied: corner in ./miniconda3/lib/python3.9/site-packages (from phoebe) (2.2.2)
Requirement already satisfied: python-socketio in ./miniconda3/lib/python3.9/site-packages (from phoebe) (4.6.1)
Requirement already satisfied: gevent in ./miniconda3/lib/python3.9/site-packages (from phoebe) (23.9.0.post1)
Requirement already satisfied: flask in ./miniconda3/lib/python3.9/site-packages (from phoebe) (2.3.3)
Requirement already satisfied: requests in ./miniconda3/lib/python3.9/site-packages (from phoebe) (2.31.0)
Requirement already satisfied: pytest in ./miniconda3/lib/python3.9/site-packages (from phoebe) (7.4.2)
Requirement already satisfied: numpy in ./miniconda3/lib/python3.9/site-packages (from phoebe) (1.25.2)
Requirement already satisfied: PyYAML>=3.13 in ./miniconda3/lib/python3.9/site-packages (from astropy->phoebe) (6.0.1)
Requirement already satisfied: pyerfa>=2.0 in ./miniconda3/lib/python3.9/site-packages (from astropy->phoebe) (2.0.0.3)
Requirement already satisfied: packaging>=19.0 in ./miniconda3/lib/python3.9/site-packages (from astropy->phoebe) (23.1)
Requirement already satisfied: matplotlib>=2.1 in ./miniconda3/lib/python3.9/site-packages (from corner->phoebe) (3.7.2)
Requirement already satisfied: cycler>=0.10 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (4.42.1)
Requirement already satisfied: pillow>=6.2.0 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (10.0.0)
Requirement already satisfied: contourpy>=1.0.1 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (1.1.0)
Requirement already satisfied: importlib-resources>=3.2.0 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (6.0.1)
Requirement already satisfied: python-dateutil>=2.7 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (1.4.5)
Requirement already satisfied: pyparsing<3.1,>=2.3.1 in ./miniconda3/lib/python3.9/site-packages (from matplotlib>=2.1->corner->phoebe) (3.0.9)
Requirement already satisfied: zipp>=3.1.0 in ./miniconda3/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib>=2.1->corner->phoebe) (3.16.2)
Requirement already satisfied: six>=1.5 in ./miniconda3/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib>=2.1->corner->phoebe) (1.16.0)
Requirement already satisfied: importlib-metadata>=3.6.0 in ./miniconda3/lib/python3.9/site-packages (from flask->phoebe) (6.8.0)
Requirement already satisfied: Jinja2>=3.1.2 in ./miniconda3/lib/python3.9/site-packages (from flask->phoebe) (3.1.2)
Requirement already satisfied: itsdangerous>=2.1.2 in ./miniconda3/lib/python3.9/site-packages (from flask->phoebe) (2.1.2)
Requirement already satisfied: Werkzeug>=2.3.7 in ./miniconda3/lib/python3.9/site-packages (from flask->phoebe) (2.3.7)
Requirement already satisfied: click>=8.1.3 in ./miniconda3/lib/python3.9/site-packages (from flask->phoebe) (8.1.7)
Requirement already satisfied: blinker>=1.6.2 in ./miniconda3/lib/python3.9/site-packages (from flask->phoebe) (1.6.2)
Requirement already satisfied: MarkupSafe>=2.0 in ./miniconda3/lib/python3.9/site-packages (from Jinja2>=3.1.2->flask->phoebe) (2.1.3)
Requirement already satisfied: python-engineio<4,>=3.13.0 in ./miniconda3/lib/python3.9/site-packages (from python-socketio->phoebe) (3.14.2)
Requirement already satisfied: zope.event in ./miniconda3/lib/python3.9/site-packages (from gevent->phoebe) (5.0)
Requirement already satisfied: zope.interface in ./miniconda3/lib/python3.9/site-packages (from gevent->phoebe) (6.0)
Requirement already satisfied: greenlet>=2.0.0 in ./miniconda3/lib/python3.9/site-packages (from gevent->phoebe) (2.0.2)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in ./miniconda3/lib/python3.9/site-packages (from pytest->phoebe) (1.1.3)
Requirement already satisfied: tomli>=1.0.0 in ./miniconda3/lib/python3.9/site-packages (from pytest->phoebe) (2.0.1)
Requirement already satisfied: iniconfig in ./miniconda3/lib/python3.9/site-packages (from pytest->phoebe) (2.0.0)
Requirement already satisfied: pluggy<2.0,>=0.12 in ./miniconda3/lib/python3.9/site-packages (from pytest->phoebe) (1.3.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./miniconda3/lib/python3.9/site-packages (from requests->phoebe) (3.2.0)
Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.9/site-packages (from requests->phoebe) (2023.7.22)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./miniconda3/lib/python3.9/site-packages (from requests->phoebe) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in ./miniconda3/lib/python3.9/site-packages (from requests->phoebe) (3.4)
Requirement already satisfied: setuptools in ./miniconda3/lib/python3.9/site-packages (from zope.event->gevent->phoebe) (68.2.0)
Building wheels for collected packages: phoebe
  Building wheel for phoebe (PEP 517) ... done
  Created wheel for phoebe: filename=phoebe-2.4.12-cp39-cp39-linux_x86_64.whl size=78325867 sha256=d54af16dfceb3a18b480e1ee16ab24569edbb6294891f76cfd0e327563f7d0de
  Stored in directory: /.cache/pip/wheels/1e/ef/ee/e831c5f696605d2f1d521aec9aa451983ffcbab5aa5be02df1
Successfully built phoebe
Installing collected packages: phoebe
  Attempting uninstall: phoebe
    Found existing installation: phoebe 2.4.10
    Uninstalling phoebe-2.4.10:
      Successfully uninstalled phoebe-2.4.10
Successfully installed phoebe-2.4.12

(base) westobin@login2:~> python3 -c "import phoebe; print(phoebe.__version__)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/miniconda3/lib/python3.9/site-packages/phoebe/__init__.py", line 377, in <module>
    from .atmospheres.passbands import install_passband, uninstall_passband, uninstall_all_passbands, download_passband, list_passband_online_history, update_passband_available, update_passband, update_all_passbands, list_all_update_passbands_available, list_online_passbands, list_installed_passbands, list_passbands, list_passband_directories, get_passband
  File "/miniconda3/lib/python3.9/site-packages/phoebe/atmospheres/passbands.py", line 22, in <module>
    import libphoebe
ImportError: /miniconda3/lib/python3.9/site-packages/libphoebe.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv

westobin avatar Jan 24 '24 14:01 westobin

Just tried phoebe 2.4.13. Unfortunately, it's still throwing the same error when importing libphoebe.

westobin avatar Apr 29 '24 17:04 westobin

@westobin, it looks like this is the same issue: https://stackoverflow.com/questions/71331756/zst28-throw-bad-array-new-lengthv-trying-to-import-c-code-in-python-after-co

It seems that c++ used for compiling and for linking are not the same. Perhaps try the workaround proposed in there, but it looks like it's a bad idea. Either way, I don't think it's phoebe; I think it's your cluster setup?

aprsa avatar Apr 29 '24 18:04 aprsa

Thanks Andrej for that link. How odd that the C++ for compiling and linking would differ (but that certainly seems to fit the symptoms of this ailment). I will try a few different things and report back.

westobin avatar Apr 30 '24 13:04 westobin

Fortunately, I was able to resolve this issue without using the workarounds. I had to downgrade GCC to 9.3.0.

Looks like I'll probably have a more significant issue when this version of GCC is no longer available since I wasn't successful with GCC 10.3.0, 11.2.0, 12.1.0, nor 12.2.0, all ending in the same error. But that's a problem for me to worry about on a different day.

I can also confirm that PHOEBE isn't the issue and 2.4.13 works just fine on my cluster and my local computer!

Thanks for the help!

westobin avatar May 30 '24 19:05 westobin