[solution] Habitat_sim installs failing in Colab

[srkiyengar]

Recently, habitat_sim colab installs seem to fail in "import habitat-sim" after successful installtion. It seems to be a complaining about missing library: ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

The two colab habitat-sim install that I have tried are,

1. https://colab.research.google.com/github/facebookresearch/habitat-sim/blob/main/examples/tutorials/colabs/ECCV_2020_Interactivity.ipynb

2. !curl -L https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s

Error details while importing habitat-sim in ECCV_2020_Interactivity.ipynb

ImportError Traceback (most recent call last) in () 17 from PIL import Image 18 ---> 19 import habitat_sim 20 from habitat_sim.utils import common as ut 21 from habitat_sim.utils import viz_utils as vut

11 frames /usr/local/lib/python3.7/dist-packages/scipy/optimize/_numdiff.py in () 6 from scipy.sparse.linalg import LinearOperator 7 from ..sparse import issparse, csc_matrix, csr_matrix, coo_matrix, find ----> 8 from ._group_columns import group_dense, group_sparse 9 10

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

Error details of "import habitat-sim" after install from "https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s"

/content/oreo-pybullet/agent_oreo.py in () ----> 1 import habitat_sim 2 import numpy as np 3 import quaternion 4 import math 5 import cv2

/usr/local/lib/python3.7/dist-packages/habitat_sim/init.py in () 11 if not getattr(builtins, "HSIM_SETUP", False): 12 # TODO: kept only for compatibiliy with existing code. Please gradually remove ---> 13 from habitat_sim import ( 14 agent, 15 attributes,

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/init.py in () 7 # type: ignore 8 ----> 9 from .agent import * # noqa: F403 10 from .controls import * # noqa: F403 11

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/agent.py in () 29 ) 30 ---> 31 from .controls import ActuationSpec, ObjectControls 32 33 all = ["ActionSpec", "SixDOFPose", "AgentState", "AgentConfiguration", "Agent"]

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/controls/init.py in () 8 from habitat_sim.agent.controls.default_controls import * # noqa: F401, F403 9 from habitat_sim.agent.controls.object_controls import ObjectControls ---> 10 from habitat_sim.agent.controls.pyrobot_noisy_controls import PyRobotNoisyActuationSpec 11 12 all = [

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/controls/pyrobot_noisy_controls.py in () 18 import magnum as mn 19 import numpy as np ---> 20 import scipy.stats 21 from attr import Attribute 22 from numpy import ndarray

/usr/local/lib/python3.7/dist-packages/scipy/stats/init.py in () 439 """ 440 --> 441 from .stats import * 442 from .distributions import * 443 from .morestats import *

/usr/local/lib/python3.7/dist-packages/scipy/stats/stats.py in () 41 import scipy.special as special 42 from scipy import linalg ---> 43 from . import distributions 44 from . import mstats_basic 45 from ._stats_mstats_common import (_find_repeats, linregress, theilslopes,

/usr/local/lib/python3.7/dist-packages/scipy/stats/distributions.py in () 6 # instead of git blame -Lxxx,+x. 7 # ----> 8 from ._distn_infrastructure import (rv_discrete, rv_continuous, rv_frozen) 9 10 from . import _continuous_distns

/usr/local/lib/python3.7/dist-packages/scipy/stats/_distn_infrastructure.py in () 22 # for root finding for continuous distribution ppf, and max likelihood 23 # estimation ---> 24 from scipy import optimize 25 26 # for functions of continuous distributions (e.g. moments, entropy, cdf)

/usr/local/lib/python3.7/dist-packages/scipy/optimize/init.py in () 398 """ 399 --> 400 from .optimize import * 401 from ._minimize import * 402 from ._root import *

/usr/local/lib/python3.7/dist-packages/scipy/optimize/optimize.py in () 34 line_search_wolfe2 as line_search, 35 LineSearchWarning) ---> 36 from ._numdiff import approx_derivative 37 from scipy._lib._util import getfullargspec_no_self as _getfullargspec 38 from scipy._lib._util import MapWrapper

/usr/local/lib/python3.7/dist-packages/scipy/optimize/_numdiff.py in () 6 from scipy.sparse.linalg import LinearOperator 7 from ..sparse import issparse, csc_matrix, csr_matrix, coo_matrix, find ----> 8 from ._group_columns import group_dense, group_sparse 9 10

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

Hopefully the details are sufficient to troubleshoot the colab environment working.The installs used to work nicely the last time I checked 3-4 weeks ago. Thanks.

[dhruvbatra]

@Skylion007 -- thoughts?

[srkiyengar]

Strings command of /usr/lib/x86_64-linux-gnu/libstdc++.so.6 for the colab install.

[arjunmajum]

Here's temporary fix that worked for me from the second solution here.

Run this one time:

!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt-get update --fix-missing
!apt-get dist-upgrade

Then, restart the runtime -- or -- modify the imports to automatically restart:

try:
  import habitat_sim
except ImportError:
  exit()

[srkiyengar]

Thank you @arjunmajum.

[Skylion007]

@Skylion007 -- thoughts?

Trying to find a long term fix here that doesn't require a Colab restart. I am really confused where the new LIBCXX is being installed / overwritten from.

[srkiyengar]

@arjunmajum's suggestion of restart worked well and was very usable. But as of last evening, executing the install script to install habitat-sim is running into another issue.

PackagesNotFoundError: The following packages are not available from current channels:

• numba=0.56.0

Any suggestion or work around will be appreciated.

---

!curl -L https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s

--2022-08-02 13:43:47-- https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh Resolving repo.continuum.io (repo.continuum.io)... 104.18.200.79, 104.18.201.79, 2606:4700::6812:c94f, ... Connecting to repo.continuum.io (repo.continuum.io)|104.18.200.79|:443... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh [following] --2022-08-02 13:43:47-- https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.130.3, 104.16.131.3, 2606:4700::6810:8203, ... Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.130.3|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 76607678 (73M) [application/x-sh] Saving to: ‘Miniconda3-latest-Linux-x86_64.sh’

Miniconda3-latest-L 100%[===================>] 73.06M 234MB/s in 0.3s

2022-08-02 13:43:48 (234 MB/s) - ‘Miniconda3-latest-Linux-x86_64.sh’ saved [76607678/76607678]

PREFIX=/usr/local Unpacking payload ... Collecting package metadata (current_repodata.json): done Solving environment: done

Package Plan

environment location: /usr/local

added / updated specs: - _libgcc_mutex==0.1=main - _openmp_mutex==4.5=1_gnu - brotlipy==0.7.0=py39h27cfd23_1003 - ca-certificates==2022.3.29=h06a4308_1 - certifi==2021.10.8=py39h06a4308_2 - cffi==1.15.0=py39hd667e15_1 - charset-normalizer==2.0.4=pyhd3eb1b0_0 - colorama==0.4.4=pyhd3eb1b0_0 - conda-content-trust==0.1.1=pyhd3eb1b0_0 - conda-package-handling==1.8.1=py39h7f8727e_0 - conda==4.12.0=py39h06a4308_0 - cryptography==36.0.0=py39h9ce1e76_0 - idna==3.3=pyhd3eb1b0_0 - ld_impl_linux-64==2.35.1=h7274673_9 - libffi==3.3=he6710b0_2 - libgcc-ng==9.3.0=h5101ec6_17 - libgomp==9.3.0=h5101ec6_17 - libstdcxx-ng==9.3.0=hd4cf53a_17 - ncurses==6.3=h7f8727e_2 - openssl==1.1.1n=h7f8727e_0 - pip==21.2.4=py39h06a4308_0 - pycosat==0.6.3=py39h27cfd23_0 - pycparser==2.21=pyhd3eb1b0_0 - pyopenssl==22.0.0=pyhd3eb1b0_0 - pysocks==1.7.1=py39h06a4308_0 - python==3.9.12=h12debd9_0 - readline==8.1.2=h7f8727e_1 - requests==2.27.1=pyhd3eb1b0_0 - ruamel_yaml==0.15.100=py39h27cfd23_0 - setuptools==61.2.0=py39h06a4308_0 - six==1.16.0=pyhd3eb1b0_1 - sqlite==3.38.2=hc218d9a_0 - tk==8.6.11=h1ccaba5_0 - tqdm==4.63.0=pyhd3eb1b0_0 - tzdata==2022a=hda174b7_0 - urllib3==1.26.8=pyhd3eb1b0_0 - wheel==0.37.1=pyhd3eb1b0_0 - xz==5.2.5=h7b6447c_0 - yaml==0.2.5=h7b6447c_0 - zlib==1.2.12=h7f8727e_1

The following NEW packages will be INSTALLED:

_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main _openmp_mutex pkgs/main/linux-64::_openmp_mutex-4.5-1_gnu brotlipy pkgs/main/linux-64::brotlipy-0.7.0-py39h27cfd23_1003 ca-certificates pkgs/main/linux-64::ca-certificates-2022.3.29-h06a4308_1 certifi pkgs/main/linux-64::certifi-2021.10.8-py39h06a4308_2 cffi pkgs/main/linux-64::cffi-1.15.0-py39hd667e15_1 charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0 colorama pkgs/main/noarch::colorama-0.4.4-pyhd3eb1b0_0 conda pkgs/main/linux-64::conda-4.12.0-py39h06a4308_0 conda-content-tru~ pkgs/main/noarch::conda-content-trust-0.1.1-pyhd3eb1b0_0 conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.8.1-py39h7f8727e_0 cryptography pkgs/main/linux-64::cryptography-36.0.0-py39h9ce1e76_0 idna pkgs/main/noarch::idna-3.3-pyhd3eb1b0_0 ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.35.1-h7274673_9 libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2 libgcc-ng pkgs/main/linux-64::libgcc-ng-9.3.0-h5101ec6_17 libgomp pkgs/main/linux-64::libgomp-9.3.0-h5101ec6_17 libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.3.0-hd4cf53a_17 ncurses pkgs/main/linux-64::ncurses-6.3-h7f8727e_2 openssl pkgs/main/linux-64::openssl-1.1.1n-h7f8727e_0 pip pkgs/main/linux-64::pip-21.2.4-py39h06a4308_0 pycosat pkgs/main/linux-64::pycosat-0.6.3-py39h27cfd23_0 pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0 pyopenssl pkgs/main/noarch::pyopenssl-22.0.0-pyhd3eb1b0_0 pysocks pkgs/main/linux-64::pysocks-1.7.1-py39h06a4308_0 python pkgs/main/linux-64::python-3.9.12-h12debd9_0 readline pkgs/main/linux-64::readline-8.1.2-h7f8727e_1 requests pkgs/main/noarch::requests-2.27.1-pyhd3eb1b0_0 ruamel_yaml pkgs/main/linux-64::ruamel_yaml-0.15.100-py39h27cfd23_0 setuptools pkgs/main/linux-64::setuptools-61.2.0-py39h06a4308_0 six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1 sqlite pkgs/main/linux-64::sqlite-3.38.2-hc218d9a_0 tk pkgs/main/linux-64::tk-8.6.11-h1ccaba5_0 tqdm pkgs/main/noarch::tqdm-4.63.0-pyhd3eb1b0_0 tzdata pkgs/main/noarch::tzdata-2022a-hda174b7_0 urllib3 pkgs/main/noarch::urllib3-1.26.8-pyhd3eb1b0_0 wheel pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0 xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0 yaml pkgs/main/linux-64::yaml-0.2.5-h7b6447c_0 zlib pkgs/main/linux-64::zlib-1.2.12-h7f8727e_1

Preparing transaction: done Executing transaction: done installation finished. WARNING: You currently have a PYTHONPATH environment variable set. This may cause unexpected behavior when running the Python interpreter in Miniconda3. For best results, please verify that your PYTHONPATH only points to directories of packages that are compatible with the Python interpreter in Miniconda3: /usr/local Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

• numba=0.56.0

Current channels:

• https://conda.anaconda.org/aihabitat/linux-64
• https://conda.anaconda.org/aihabitat/noarch
• https://conda.anaconda.org/conda-forge/linux-64
• https://conda.anaconda.org/conda-forge/noarch
• https://repo.anaconda.com/pkgs/main/linux-64
• https://repo.anaconda.com/pkgs/main/noarch
• https://repo.anaconda.com/pkgs/r/linux-64
• https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're looking for, navigate to

https://anaconda.org/

and use the search bar at the top of the page.

An error occured during the installation of Habitat-sim or Habitat-Lab.

---

[pkachana3]

@srkiyengar Were you able to get this installation issue resolved? I am having the same problem as well.

[srkiyengar]

Not properly @pkachana3. "conda search -f numba" stops at numba version 0.55.1 where as the colab default seems to be 0.56.0

I harded coded numba to 0.51.2. In the install script:

conda install -S -y --prefix /usr/local -c "${CHANNEL}" -c conda-forge habitat-sim=0.2.0 headless withbullet "python=${PYTHON_VERSION}" "numpy=${NUMPY_VERSION}" "pillow=${PIL_VERSION}" "cffi=${CFFI_VERSION}" "scipy=${SCIPY_VERSION}" "numba=0.51.2"

It seems to install the habitat.

[aclegg3]

Thanks for the temporary solutions here, folks! With these I'm able to run the colabs successfully. @Skylion007 and I will work on a real fix.

For now, you can:

1. modify the install cell to use the installer on a branch with the change suggested by @srkiyengar:

https://raw.githubusercontent.com/facebookresearch/habitat-sim/colab-numba-version/examples/colab_utils/colab_install.sh

Edited path 10/11

2. apply the fix suggested by @arjunmajum in this comment:

Run this one time:

!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt-get update --fix-missing
!apt-get dist-upgrade

Then, restart the runtime

[rpartsey]

@aclegg3 thank you for advice! works for me

[aclegg3]

Note we have stopped supporting colab and converted tutorials to local Jupyter notebooks instead.