activity-browser icon indicating copy to clipboard operation
activity-browser copied to clipboard

LCA calculation hangs. Debug window shows error : ValueError: Shape of passed values is (3, 1), indices imply (3, 16)

Open raphaeljolivet opened this issue 3 years ago • 1 comments

When doing simple calculation of 1 activity and 1 impact method, the calculation hangs ("waiting"Cursor). In the debug windows I get the following stack trace :

Reset project settings directory to: /home/rjolivet/.local/share/Brightway3/incer-acv-app.57d1238e6c85dd8d8b3701547605ddd1
Reset metadata.
Loaded project: incer-acv-app
Current shape and databases in the MetaDataStore: (0, 0) set()
Adding: incer-acv-app
+showing tab: Activity Details
+showing tab: Activity Details
+showing tab: Activity Details
adding graph tab
New Graph for key:  ('incer-acv-app', 'Full pv installation')
+showing tab: Graph Explorer
Current shape and databases in the MetaDataStore: (21, 17) {'incer-acv-app'}
Adding: ecoinvent 3.4 cut off
Adding: biosphere3
Traceback (most recent call last):
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tab.py", line 58, in generate_setup
    new_tab = LCAResultsSubTab(data, self)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tabs.py", line 111, in __init__
    ef=ElementaryFlowContributionTab(self),
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tabs.py", line 881, in __init__
    self.toggle_comparisons(self.switches.indexes.func)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tabs.py", line 750, in toggle_comparisons
    self.update_tab()
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tabs.py", line 820, in update_tab
    self.set_combobox_changes()
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tabs.py", line 801, in set_combobox_changes
    self.df = self.update_dataframe(**compare_fields)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/layouts/tabs/LCA_results_tabs.py", line 890, in update_dataframe
    return self.parent.contributions.top_elementary_flow_contributions(
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/bwutils/multilca.py", line 720, in top_elementary_flow_contributions
    labelled_df = self.get_labelled_contribution_dict(
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/bwutils/multilca.py", line 508, in get_labelled_contribution_dict
    df = pd.concat(dfs, sort=False, axis=1)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/pandas/util/_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 347, in concat
    op = _Concatenator(
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/pandas/core/reshape/concat.py", line 401, in __init__
    objs = list(objs)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/activity_browser/bwutils/multilca.py", line 505, in <genexpr>
    pd.DataFrame(v.values(), index=list(v.keys()), columns=[k])
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/pandas/core/frame.py", line 737, in __init__
    mgr = ndarray_to_mgr(
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 351, in ndarray_to_mgr
    _check_values_indices_shape_match(values, index, columns)
  File "/home/rjolivet/mambaforge/envs/ab/lib/python3.9/site-packages/pandas/core/internals/construction.py", line 422, in _check_values_indices_shape_match
    raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")
ValueError: Shape of passed values is (3, 1), indices imply (3, 16)

raphaeljolivet avatar Aug 04 '22 13:08 raphaeljolivet

Thanks for sharing the issue.

Could you provide the following information please?:

  • Screenshot of the tab LCA setup
  • If it doesn't contain anything confidential, the problematic database (export is possible through top left file > export Database .. > choose excel format and a sensible location.
  • The output of conda list, which will be a long lost of libraries AB uses.

Regarding AB hanging (the loading cursor not resetting after failed calculation), that's a known issue: #767. The icon will not reset until AB is restarted, but you can still use AB after a calculation fails.

marc-vdm avatar Aug 04 '22 13:08 marc-vdm

I get exactly the same error message for doing a scenarioLCA calculation with a new DB relinked (eco_export_SS) to a superstructure from premise. Conducting a scenarioLCA just with the superstructure itself works in the AB.

What is interesting is that the new DB (eco_export_SS) does work for scenario LCAcalculations using BW2, so the DB seems to be functional for BW2.

LCA setup grafik

activity-browser-dev      2022.11.16                 py_0    bsteubing
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
arrow                     1.2.3              pyhd8ed1ab_0    conda-forge
asteval                   0.9.27             pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.11.1             pyha770c72_0    conda-forge
brightway2                2.4.2                      py_5    cmutel
brotli                    1.0.9                hcfcfb64_8    conda-forge
brotli-bin                1.0.9                hcfcfb64_8    conda-forge
brotli-python             1.0.9            py39h99910a6_8    conda-forge
brotlicffi                1.0.9.2          py39h99910a6_3    conda-forge
brotlipy                  0.7.0           py39ha55989b_1005    conda-forge
bw2analyzer               0.10                       py_1    cmutel
bw2calc                   1.8.1                      py_2    cmutel
bw2data                   3.6.5                      py_0    cmutel
bw2io                     0.8.7                      py_0    cmutel
bw2parameters             0.7                        py_0    cmutel
bw_migrations             0.1                        py_0    cmutel
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2022.9.24            h5b45459_0    conda-forge
certifi                   2022.9.24          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1           py39h68f70e3_2    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.6            py39h1f6ef14_0    conda-forge
cryptography              38.0.3           py39h58e9bdb_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
dill                      0.3.6              pyhd8ed1ab_1    conda-forge
docopt                    0.6.2                      py_1    conda-forge
eidl                      1.5.1                      py_0    bsteubing
eight                     1.0.0                      py_0    conda-forge
et_xmlfile                1.0.1                   py_1001    conda-forge
fasteners                 0.17.3             pyhd8ed1ab_0    conda-forge
fonttools                 4.38.0           py39ha55989b_1    conda-forge
freetype                  2.12.1               h546665d_0    conda-forge
future                    0.18.2             pyhd8ed1ab_6    conda-forge
gettext                   0.21.1               h5728263_0    conda-forge
glib                      2.74.1               h12be248_1    conda-forge
glib-tools                2.74.1               h12be248_1    conda-forge
gst-plugins-base          1.20.3               h001b923_2    conda-forge
gstreamer                 1.20.3               h6b5321d_2    conda-forge
icu                       70.1                 h0e60522_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        5.0.0              pyha770c72_1    conda-forge
importlib_metadata        5.0.0                hd8ed1ab_1    conda-forge
inflate64                 0.3.1            py39ha55989b_2    conda-forge
intel-openmp              2022.1.0          h57928b3_3787    conda-forge
jpeg                      9e                   h8ffe710_2    conda-forge
kiwisolver                1.4.4            py39h1f6ef14_1    conda-forge
krb5                      1.19.3               h1176d77_0    conda-forge
lcms2                     2.14                 h90d422f_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libblas                   3.9.0              16_win64_mkl    conda-forge
libbrotlicommon           1.0.9                hcfcfb64_8    conda-forge
libbrotlidec              1.0.9                hcfcfb64_8    conda-forge
libbrotlienc              1.0.9                hcfcfb64_8    conda-forge
libcblas                  3.9.0              16_win64_mkl    conda-forge
libclang                  14.0.6          default_h77d9078_0    conda-forge
libclang13                14.0.6          default_h77d9078_0    conda-forge
libdeflate                1.14                 hcfcfb64_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libglib                   2.74.1               he8f3873_1    conda-forge
libiconv                  1.17                 h8ffe710_0    conda-forge
liblapack                 3.9.0              16_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.38               h19919ed_0    conda-forge
libsqlite                 3.39.4               hcfcfb64_0    conda-forge
libtiff                   4.4.0                h8e97e67_4    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.2.4                h8ffe710_0    conda-forge
libwebp-base              1.2.4                h8ffe710_0    conda-forge
libxcb                    1.13              hcd874cb_1004    conda-forge
libxml2                   2.10.3               hc3477c8_0    conda-forge
libxslt                   1.1.37               h0192164_0    conda-forge
libzlib                   1.2.13               hcfcfb64_4    conda-forge
lxml                      4.9.1            py39h0942119_1    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
matplotlib-base           3.6.2            py39haf65ace_0    conda-forge
mkl                       2022.1.0           h6a75c08_874    conda-forge
mrio_common_metadata      0.2.1                      py_0    cmutel
msys2-conda-epoch         20160418                      1    conda-forge
multiprocess              0.70.14          py39ha55989b_3    conda-forge
multivolumefile           0.2.3              pyhd8ed1ab_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
networkx                  2.8.8              pyhd8ed1ab_0    conda-forge
numpy                     1.23.4           py39hbccbffa_1    conda-forge
openjpeg                  2.5.0                hc9384bd_1    conda-forge
openpyxl                  3.0.10           py39ha55989b_2    conda-forge
openssl                   1.1.1s               hcfcfb64_0    conda-forge
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.5.1            py39h2ba5b7c_1    conda-forge
pathos                    0.3.0              pyhd8ed1ab_0    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                h17e33f8_0    conda-forge
peewee                    3.15.3           py39habf9222_1    conda-forge
pillow                    9.2.0            py39h595c93f_3    conda-forge
pip                       22.3.1             pyhd8ed1ab_0    conda-forge
pox                       0.3.2              pyhd8ed1ab_0    conda-forge
ppft                      1.7.6.6            pyhd8ed1ab_0    conda-forge
psutil                    5.9.4            py39ha55989b_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
py7zr                     0.20.2             pyhd8ed1ab_0    conda-forge
pybcj                     1.0.1            py39ha55989b_2    conda-forge
pybcpy                    0.0.17             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycryptodomex             3.15.0           py39ha55989b_1    conda-forge
pyopenssl                 22.1.0             pyhd8ed1ab_0    conda-forge
pypardiso                 0.4.1              pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyperclip                 1.8.2              pyhd8ed1ab_2    conda-forge
pyppmd                    1.0.0            py39h99910a6_1    conda-forge
pyprind                   2.11.2          py39hcbf5309_1004    conda-forge
pyside2                   5.15.6           py39h56cbfbc_3    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.9.13          h9a09f29_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.9                      2_cp39    conda-forge
pytz                      2022.6             pyhd8ed1ab_0    conda-forge
pyxlsb                    1.0.10             pyhd8ed1ab_0    conda-forge
pyzstd                    0.15.3           py39h95af829_1    conda-forge
qt-main                   5.15.6               hf0cf448_0    conda-forge
qt-webengine              5.15.4               h7325c83_3    conda-forge
requests                  2.28.1             pyhd8ed1ab_1    conda-forge
salib                     1.4.6.1            pyhd8ed1ab_0    conda-forge
scipy                     1.9.3            py39hfbf2dce_1    conda-forge
seaborn                   0.12.1               hd8ed1ab_0    conda-forge
seaborn-base              0.12.1             pyhd8ed1ab_0    conda-forge
setuptools                65.5.1             pyhd8ed1ab_0    conda-forge
setuptools-scm            7.0.5              pyhd8ed1ab_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
sqlite                    3.39.4               hcfcfb64_0    conda-forge
stats_arrays              0.6.5                      py_2    cmutel
statsmodels               0.13.5           py39hc266a54_2    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tbb                       2021.6.0             h91493d7_1    conda-forge
texttable                 1.6.4              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
typing-extensions         4.4.0                hd8ed1ab_0    conda-forge
typing_extensions         4.4.0              pyha770c72_0    conda-forge
tzdata                    2022f                h191b570_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
unicodecsv                0.14.1                     py_1    conda-forge
unicodedata2              15.0.0           py39ha55989b_0    conda-forge
unidecode                 1.3.6              pyhd8ed1ab_0    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
vc                        14.3                 h3d8a991_9    conda-forge
voluptuous                0.13.1             pyhd8ed1ab_0    conda-forge
vs2015_runtime            14.32.31332          h1d6e394_9    conda-forge
wheel                     0.38.2             pyhd8ed1ab_0    conda-forge
whoosh                    2.7.4            py39hcbf5309_7    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
wrapt                     1.14.1           py39ha55989b_1    conda-forge
xlrd                      2.0.1              pyhd8ed1ab_3    conda-forge
xlsxwriter                3.0.3              pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
zipfile-deflate64         0.2.0            py39ha55989b_3    conda-forge
zipp                      3.10.0             pyhd8ed1ab_0    conda-forge
zstd                      1.5.2                h7755175_4    conda-forge

The used DB contains only 2 processes (see excel file), but is linked to a superstructure (which I could share in addition, if required) DB_eco_export_SS.xlsx

Error message:

Traceback (most recent call last):
  File "...lib\site-packages\activity_browser\layouts\tabs\LCA_results_tab.py", line 56, in generate_setup
    new_tab = LCAResultsSubTab(data, self)
  File "...\lib\site-packages\activity_browser\layouts\tabs\LCA_results_tabs.py", line 111, in __init__
    ef=ElementaryFlowContributionTab(self),
  File "...\lib\site-packages\activity_browser\layouts\tabs\LCA_results_tabs.py", line 874, in __init__
    self.toggle_comparisons(self.switches.indexes.func)
  File "...\lib\site-packages\activity_browser\layouts\tabs\LCA_results_tabs.py", line 750, in toggle_comparisons
    self.update_tab()
  File "...lib\site-packages\activity_browser\layouts\tabs\LCA_results_tabs.py", line 813, in update_tab
    self.set_combobox_changes()
  File "...lib\site-packages\activity_browser\layouts\tabs\LCA_results_tabs.py", line 801, in set_combobox_changes
    self.df = self.update_dataframe(**compare_fields)
  File "...lib\site-packages\activity_browser\layouts\tabs\LCA_results_tabs.py", line 883, in update_dataframe
    return self.parent.contributions.top_elementary_flow_contributions(
  File "...\lib\site-packages\activity_browser\bwutils\multilca.py", line 788, in top_elementary_flow_contributions
    labelled_df = self.get_labelled_contribution_dict(
  File "...\lib\site-packages\activity_browser\bwutils\multilca.py", line 559, in get_labelled_contribution_dict
    df = pd.concat(dfs, sort=False, axis=1)
  File "...\lib\site-packages\pandas\util\_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "...\lib\site-packages\pandas\core\reshape\concat.py", line 368, in concat
    op = _Concatenator(
  File "...lib\site-packages\pandas\core\reshape\concat.py", line 422, in __init__
    objs = list(objs)
  File "...\site-packages\activity_browser\bwutils\multilca.py", line 556, in <genexpr>
    pd.DataFrame(v.values(), index=list(v.keys()), columns=[k])
  File "...\lib\site-packages\pandas\core\frame.py", line 760, in __init__
    mgr = ndarray_to_mgr(
  File "...\lib\site-packages\pandas\core\internals\construction.py", line 349, in ndarray_to_mgr
    _check_values_indices_shape_match(values, index, columns)
  File "...\lib\site-packages\pandas\core\internals\construction.py", line 420, in _check_values_indices_shape_match
    raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")
ValueError: Shape of passed values is (5, 1), indices imply (5, 8)

CHarpprecht avatar Feb 13 '23 10:02 CHarpprecht

In my case, I could resolve the problem: My DB was created and modified with BW before I wrote it to disk. Within the bw2-dict of the db, I added the keys "production volume" for the activities. This is also visible as a column in the excel sheet of the AB-export of the DB. It seems that the AB cannot handle this additional information. Removing the key from the bw2-dict before writing the DB made the DB functional for scenarioLCA in the AB.

CHarpprecht avatar Feb 13 '23 11:02 CHarpprecht

Closing this as resolved in #1025

marc-vdm avatar Sep 14 '23 14:09 marc-vdm