_setup_buf conflicting Dict types when using Gymnasium with FetchReach environment

[devidduma]

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, gym, torch, numpy, sys
  print(tianshou.__version__, gym.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)
  0.4.11 0.26.2 1.13.1+cpu 1.23.5 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] win32

Hi,

I am trying to run examples/mujoco/fetch_her_ddpg.py.

After the first run, the following error is thrown:

gym.error.NameNotFound: Environment FetchReach doesn't exist. 

So I changed:

import gym

into:

import gymnasium as gym

Now I get: AttributeError: 'NoneType' object has no attribute 'type' with the following stacktrace:

Traceback (most recent call last):
File "C:\Users\devid\PycharmProjects\thesis\examples\fetch_her_ddpg.py", line 229, in <module>
  test_ddpg()
File "C:\Users\devid\PycharmProjects\thesis\examples\fetch_her_ddpg.py", line 115, in test_ddpg
  env, train_envs, test_envs = make_fetch_env(
File "C:\Users\devid\PycharmProjects\thesis\examples\fetch_her_ddpg.py", line 81, in make_fetch_env
  train_envs = ShmemVectorEnv(
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\venvs.py", line 412, in __init__
  super().__init__(env_fns, worker_fn, **kwargs)
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\venvs.py", line 80, in __init__
  self.workers = [worker_fn(fn) for fn in env_fns]
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\venvs.py", line 80, in <listcomp>
  self.workers = [worker_fn(fn) for fn in env_fns]
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\venvs.py", line 410, in worker_fn
  return SubprocEnvWorker(fn, share_memory=True)
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\worker\subproc.py", line 152, in __init__
  self.buffer = _setup_buf(obs_space)
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\worker\subproc.py", line 61, in _setup_buf
  return ShArray(space.dtype, space.shape)  # type: ignore
File "C:\Users\devid\PycharmProjects\thesis\venv\lib\site-packages\tianshou\env\worker\subproc.py", line 37, in __init__
  self.arr = Array(_NP_TO_CT[dtype.type], int(np.prod(shape)))  # type: ignore
AttributeError: 'NoneType' object has no attribute 'type'

Implementation of _setup_buf() function:

def _setup_buf(space: gym.Space) -> Union[dict, tuple, ShArray]:
  if isinstance(space, gym.spaces.Dict):
      assert isinstance(space.spaces, OrderedDict)
      return {k: _setup_buf(v) for k, v in space.spaces.items()}
  elif isinstance(space, gym.spaces.Tuple):
      assert isinstance(space.spaces, tuple)
      return tuple([_setup_buf(t) for t in space.spaces])
  else:
      return ShArray(space.dtype, space.shape)  # type: ignore

While debugging, the actual data type for space: gym.Space is gymnasium.spaces.dict.Dict. The _setup_buf() function is expecting gym.spaces.Dict, so the first conditional is evaluated to False instead of True. On the other hand, sticking to:

import gym

does not seem to be an option, because of the error message being thrown in the beginning.

How can I tackle this?

[Trinkle23897]

Try this locally via pip install -e .?

def _setup_buf(space: gym.Space) -> Union[dict, tuple, ShArray]:
-  if isinstance(space, gym.spaces.Dict):
+  if isinstance(space, (gym.spaces.Dict, gymnasium.spaces.Dict)):
      assert isinstance(space.spaces, OrderedDict)
      return {k: _setup_buf(v) for k, v in space.spaces.items()}
  elif isinstance(space, gym.spaces.Tuple):
      assert isinstance(space.spaces, tuple)
      return tuple([_setup_buf(t) for t in space.spaces])
  else:
      return ShArray(space.dtype, space.shape)  # type: ignore

btw there is a pending pr #789 that will resolve all the issues related to gym/gymnasium inconsistency

[devidduma]

You are right, editing the source code is a good option.

I just tried pull request #789 and it works great. I am glad that the next update will fully integrate gymnasium.

Thank you very much!