ibc
ibc copied to clipboard
Published
20 hours ago •
google-research
google-research
Unit tests fail
Hi, it seems that the unit tests do not work out of the box. I'm working in a clean conda environment with Python 3.7.13. All of the prerequisites are installed with the versions described in the readme, as well as CUDA and cuDNN (Tensorflow has GPU access).
Here's the complete output from the test script:
Test script output
ibc-test ❯ ./ibc/run_tests.sh
bash: /home/arc/miniconda3/envs/ibc-test/lib/libtinfo.so.6: no version information available (required by bash)
Running run_tests.sh in directory /home/arc/noah
Running tests: /home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py
/home/arc/noah/ibc/environments/block_pushing/block_pushing_test.py
/home/arc/noah/ibc/environments/utils/utils_pybullet_test.py
/home/arc/noah/ibc/environments/utils/xarm_sim_robot_test.py
/home/arc/noah/ibc/environments/particle/particle_test.py
/home/arc/noah/ibc/ibc/agents/mcmc_test.py
/home/arc/noah/ibc/ibc/train/stats_test.py
/home/arc/noah/ibc/data/dataset_test.py
***********************************************************************
Running test /home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py
***********************************************************************
PYTHONPATH=:/home/arc/noah:/home/arc/noah/ibc/.. python3 /home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py --alsologtostderr
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/keras_preprocessing/image/utils.py:23: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': pil_image.NEAREST,
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/keras_preprocessing/image/utils.py:24: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': pil_image.BILINEAR,
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/keras_preprocessing/image/utils.py:25: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': pil_image.BICUBIC,
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/keras_preprocessing/image/utils.py:28: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
if hasattr(pil_image, 'HAMMING'):
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/keras_preprocessing/image/utils.py:30: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
if hasattr(pil_image, 'BOX'):
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/keras_preprocessing/image/utils.py:33: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
if hasattr(pil_image, 'LANCZOS'):
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/__init__.py:56: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if (distutils.version.LooseVersion(tf_version) <
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tensorflow_probability/python/__init__.py:61: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if (distutils.version.LooseVersion(tf.__version__) <
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/utils/common.py:87: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
and (distutils.version.LooseVersion(tf.__version__) <=
pybullet build time: Jun 28 2022 14:19:23
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/envs/registration.py:416: UserWarning: WARN: The `registry.env_specs` property along with `EnvSpecTree` is deprecated. Please use `registry` directly as a dictionary instead.
"The `registry.env_specs` property along with `EnvSpecTree` is deprecated. Please use `registry` directly as a dictionary instead."
2022-06-29 13:44:14.864729: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-29 13:44:14.868489: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-29 13:44:14.868817: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
Running tests under Python 3.7.13: /home/arc/miniconda3/envs/ibc-test/bin/python3
[ RUN ] Blocks2DTest.test_load_push_env
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/spaces/box.py:112: UserWarning: WARN: Box bound precision lowered by casting to float32
logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
argv[0]=
I0629 13:44:14.880214 140609892135296 utils_pybullet.py:85] Loading URDF plane.urdf
I0629 13:44:14.885888 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/workspace.urdf
I0629 13:44:14.886190 140609892135296 utils_pybullet.py:85] Loading URDF xarm/xarm6_robot.urdf
I0629 13:44:14.908028 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/suction/suction-head-long.urdf
I0629 13:44:14.911714 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/zone.urdf
I0629 13:44:14.912028 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/zone2.urdf
I0629 13:44:14.912325 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/block.urdf
I0629 13:44:14.912554 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/block2.urdf
INFO:tensorflow:time(__main__.Blocks2DTest.test_load_push_env): 0.09s
I0629 13:44:14.959522 140609892135296 test_util.py:2189] time(__main__.Blocks2DTest.test_load_push_env): 0.09s
[ OK ] Blocks2DTest.test_load_push_env
[ RUN ] Blocks2DTest.test_serialize_state_push
argv[0]=
I0629 13:44:14.963971 140609892135296 utils_pybullet.py:85] Loading URDF plane.urdf
I0629 13:44:14.969090 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/workspace.urdf
I0629 13:44:14.969393 140609892135296 utils_pybullet.py:85] Loading URDF xarm/xarm6_robot.urdf
I0629 13:44:14.987162 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/suction/suction-head-long.urdf
I0629 13:44:14.991161 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/zone.urdf
I0629 13:44:14.991568 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/zone2.urdf
I0629 13:44:14.991911 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/block.urdf
I0629 13:44:14.992150 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/block2.urdf
INFO:tensorflow:time(__main__.Blocks2DTest.test_serialize_state_push): 0.13s
I0629 13:44:15.085130 140609892135296 test_util.py:2189] time(__main__.Blocks2DTest.test_serialize_state_push): 0.13s
[ OK ] Blocks2DTest.test_serialize_state_push
[ RUN ] Blocks2DTest.test_session
[ SKIPPED ] Blocks2DTest.test_session
[ RUN ] Blocks2DTest.test_validate_environment
argv[0]=
I0629 13:44:15.090482 140609892135296 utils_pybullet.py:85] Loading URDF plane.urdf
I0629 13:44:15.095655 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/workspace.urdf
I0629 13:44:15.095959 140609892135296 utils_pybullet.py:85] Loading URDF xarm/xarm6_robot.urdf
I0629 13:44:15.113764 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/suction/suction-head-long.urdf
I0629 13:44:15.117837 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/zone.urdf
I0629 13:44:15.118308 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/zone2.urdf
I0629 13:44:15.118738 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/block.urdf
I0629 13:44:15.119005 140609892135296 utils_pybullet.py:85] Loading URDF ibc/environments/assets/block2.urdf
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py:98: UserWarning: WARN: We recommend you to use a symmetric and normalized Box action space (range=[-1, 1]) https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html
"We recommend you to use a symmetric and normalized Box action space (range=[-1, 1]) "
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py:217: UserWarning: WARN: Future gym versions will require that `Env.reset` can be passed a `seed` instead of using `Env.seed` for resetting the environment random number generator.
"Future gym versions will require that `Env.reset` can be passed a `seed` instead of using `Env.seed` for resetting the environment random number generator. "
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py:229: UserWarning: WARN: Future gym versions will require that `Env.reset` can be passed `return_info` to return information from the environment resetting.
"Future gym versions will require that `Env.reset` can be passed `return_info` to return information from the environment resetting."
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py:234: UserWarning: WARN: Future gym versions will require that `Env.reset` can be passed `options` to allow the environment initialisation to be passed additional information.
"Future gym versions will require that `Env.reset` can be passed `options` to allow the environment initialisation to be passed additional information."
/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/spaces/box.py:197: UserWarning: WARN: Casting input x to numpy array.
logger.warn("Casting input x to numpy array.")
INFO:tensorflow:time(__main__.Blocks2DTest.test_validate_environment): 0.09s
I0629 13:44:15.179256 140609892135296 test_util.py:2189] time(__main__.Blocks2DTest.test_validate_environment): 0.09s
[ FAILED ] Blocks2DTest.test_validate_environment
======================================================================
FAIL: test_validate_environment (__main__.Blocks2DTest)
Blocks2DTest.test_validate_environment
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py", line 34, in test_validate_environment
utils.validate_py_environment(env)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/utils.py", line 75, in validate_py_environment
time_step = environment.reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 196, in reset
self._current_time_step = self._reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/wrappers.py", line 111, in _reset
return self._env.reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 196, in reset
self._current_time_step = self._reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/gym_wrapper.py", line 193, in _reset
observation = self._gym_env.reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/time_limit.py", line 66, in reset
return self.env.reset(**kwargs)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
return self.env.reset(**kwargs)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/env_checker.py", line 47, in reset
return passive_env_reset_check(self.env, **kwargs)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py", line 247, in passive_env_reset_check
_check_obs(obs, env.observation_space, "reset")
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py", line 115, in _check_obs
), f"{pre} is not contained with the observation space ({observation_space})"
AssertionError: The observation returned by the `reset()` method is not contained with the observation space (Dict(block_translation: Box(-5.0, 5.0, (2,), float32), block_orientation: Box(-6.2831855, 6.2831855, (1,), float32), block2_translation: Box(-5.0, 5.0, (2,), float32), block2_orientation: Box(-6.2831855, 6.2831855, (1,), float32), effector_translation: Box([ 0.05 -0.6 ], [0.8 0.6], (2,), float32), effector_target_translation: Box([ 0.05 -0.6 ], [0.8 0.6], (2,), float32), target_translation: Box(-5.0, 5.0, (2,), float32), target_orientation: Box(-6.2831855, 6.2831855, (1,), float32), target2_translation: Box(-5.0, 5.0, (2,), float32), target2_orientation: Box(-6.2831855, 6.2831855, (1,), float32)))
----------------------------------------------------------------------
Ran 4 tests in 0.310s
FAILED (failures=1, skipped=1)
ERROR: 'PYTHONPATH=:/home/arc/noah:/home/arc/noah/ibc/.. python3 /home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py --alsologtostderr' failed!
Here's the last part of that formatted a bit more nicely:
INFO:tensorflow:time(__main__.Blocks2DTest.test_validate_environment): 0.09s
I0629 13:44:15.179256 140609892135296 test_util.py:2189] time(__main__.Blocks2DTest.test_validate_environment): 0.09s
[ FAILED ] Blocks2DTest.test_validate_environment
======================================================================
FAIL: test_validate_environment (__main__.Blocks2DTest)
Blocks2DTest.test_validate_environment
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py", line 34, in test_validate_environment
utils.validate_py_environment(env)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/utils.py", line 75, in validate_py_environment
time_step = environment.reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 196, in reset
self._current_time_step = self._reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/wrappers.py", line 111, in _reset
return self._env.reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 196, in reset
self._current_time_step = self._reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/gym_wrapper.py", line 193, in _reset
observation = self._gym_env.reset()
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/time_limit.py", line 66, in reset
return self.env.reset(**kwargs)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/order_enforcing.py", line 42, in reset
return self.env.reset(**kwargs)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/env_checker.py", line 47, in reset
return passive_env_reset_check(self.env, **kwargs)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py", line 247, in passive_env_reset_check
_check_obs(obs, env.observation_space, "reset")
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py", line 115, in _check_obs
), f"{pre} is not contained with the observation space ({observation_space})"
AssertionError: The observation returned by the `reset()` method is not contained with the observation space (Dict(block_translation: Box(-5.0, 5.0, (2,), float32), block_orientation: Box(-6.2831855, 6.2831855, (1,), float32), block2_translation: Box(-5.0, 5.0, (2,), float32), block2_orientation: Box(-6.2831855, 6.2831855, (1,), float32), effector_translation: Box([ 0.05 -0.6 ], [0.8 0.6], (2,), float32), effector_target_translation: Box([ 0.05 -0.6 ], [0.8 0.6], (2,), float32), target_translation: Box(-5.0, 5.0, (2,), float32), target_orientation: Box(-6.2831855, 6.2831855, (1,), float32), target2_translation: Box(-5.0, 5.0, (2,), float32), target2_orientation: Box(-6.2831855, 6.2831855, (1,), float32)))
----------------------------------------------------------------------
The issue seems to be that several fields of the observation returned by BlockPushMultimodal._compute_state() need to be converted to np arrays with dtype np.float32. After doing that and running the test again, I get the following error instead:
INFO:tensorflow:time(__main__.Blocks2DTest.test_validate_environment): 0.1s
I0629 14:04:36.733099 140095945187712 test_util.py:2189] time(__main__.Blocks2DTest.test_validate_environment): 0.1s
[ FAILED ] Blocks2DTest.test_validate_environment
======================================================================
ERROR: test_validate_environment (__main__.Blocks2DTest)
Blocks2DTest.test_validate_environment
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/arc/noah/ibc/environments/block_pushing/block_pushing_multimodal_test.py", line 34, in test_validate_environment
utils.validate_py_environment(env)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/utils.py", line 84, in validate_py_environment
time_step = environment.step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 233, in step
self._current_time_step = self._step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/wrappers.py", line 117, in _step
time_step = self._env.step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 233, in step
self._current_time_step = self._step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/tf_agents/environments/gym_wrapper.py", line 215, in _step
observation, reward, self._done, self._info = self._gym_env.step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/time_limit.py", line 49, in step
observation, reward, done, info = self.env.step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/order_enforcing.py", line 37, in step
return self.env.step(action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/wrappers/env_checker.py", line 39, in step
return passive_env_step_check(self.env, action)
File "/home/arc/miniconda3/envs/ibc-test/lib/python3.7/site-packages/gym/utils/passive_env_checker.py", line 273, in passive_env_step_check
if np.any(np.isnan(obs)):
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
----------------------------------------------------------------------
This is the same error as in #14 so I'd guess these issues are related.
Any thoughts?
Hi there, thanks. Especially, thanks for confirming that all of your package versions match what's in the README.
This issue didn't used to exist, and the code in this repo hasn't changed, so it seems likely due to some dependency shifting underneath us in a way that isn't captured by just the package versions.
Hi @noahcgreen, so I just ran a test on my side, and my tests are passing. My computer though had all of these package versions installed from before.
Could you share your output of pip list? I wonder if you are asking for certain pip versions but they are not all getting installed as asked for?