PyEnvironment Methods Incompatible with TF

[ergsfe]

The docstring for tf_py_environment.__getattr__ indicates that certain PyEnvironment methods might be incompatible with TF.

def __getattr__(self, name: Text) -> Any:
    """Enables access attributes of the wrapped PyEnvironment.
    Use with caution since methods of the PyEnvironment can be incompatible
    with TF.
    Args:
      name: Name of the attribute.
    Returns:
      The attribute.
    """
    if name in self.__dict__:
      return getattr(self, name)
    return getattr(self._env, name)

What makes a Py Environment method incompatible with tensorflow?

I ran across this issue when trying to call .seed on the tf_py_environment. I implemented a .seed function for my subclass of py_environment, but calling .seed on the wrapper doesn't lead to the .seed function of the subclass being called. Perhaps this is intentional?

[egonina]

cc @kbanoop, can you PTAL at this question about envs? Thanks!

[kbanoop]

Hi, This docstring is just meant to indicate that the methods in PyEnvironment do not accept/return Tensors or do other things that are not TF compatible.

But calling .seed on TFPyEnvironment should call .seed on self._env, whether that function is compatible or not. If that is not happening, it is likely a bug. What happens if you print(wrapper.seed)?

[ergsfe]

import tensorflow as tf
import numpy as np
import tf_agents
import tf_agents.environments.py_environment
import tf_agents.environments.tf_py_environment
import tf_agents.specs
class ExamplePyEnv(tf_agents.environments.py_environment.PyEnvironment):
    def __init__(self):
        super().__init__()
    def _step(self,action):
        return np.array([0.0])
    def _reset(self,action):
        return np.array([0.0])
    def seed(self,seed):
        return seed
    def action_spec(self):
        return tf_agents.specs.ArraySpec(shape=(),dtype=np.int32)
    def observation_spec(self):
        return tf_agents.specs.ArraySpec(shape=(1,),dtype=np.float32)
epe = ExamplePyEnv()
epe.seed(50)
tfpyenv = tf_agents.environments.tf_py_environment.TFPyEnvironment(epe)
try:
    tfpyenv.seed(51)
except Exception as e:
    print(repr(e))
print(tfpyenv.seed)

NotImplementedError('No seed support for this environment.')
<bound method PyEnvironment.seed of <tf_agents.environments.batched_py_environment.BatchedPyEnvironment object at 0x0000018E2E8A6BE0>>

[ebrevdo]

A full stack trace would be more useful, but I think the issue comes from the fact that we wrap the env in a BatchedPyEnv, for which we haven't implemented batched seeding. Can you provide the full stack trace to confirm?

[ergsfe]

I think that the .seed function is called on the BatchedPyEnv, which doesn't implement it.

NotImplementedError('No seed support for this environment.')
Traceback (most recent call last):
  File "<ipython-input-1-eee22a88ac7c>", line 26, in <module>
    tfpyenv.seed(51)
  File "d:\python\env\lib\site-packages\tf_agents\environments\py_environment.py", line 260, in seed
    raise NotImplementedError('No seed support for this environment.')
NotImplementedError: No seed support for this environment.

<bound method PyEnvironment.seed of <tf_agents.environments.batched_py_environment.BatchedPyEnvironment object at 0x00000237A2DBF340>>

[ebrevdo]

Would you be interested in submitting a PR adding BatchedPyEnvironment seed setters and getters that accept/return lists of seeds?

[ergsfe]

I can try doing that.