maddpg-rllib
maddpg-rllib copied to clipboard
fixes import from rllib
maddpg is currently in rllib.contrib rather than rllib.agents
see https://github.com/ray-project/ray/issues/6476
Hey, when I try running this PR to reproduce the results of the paper, I still get related issues, error log below. Given that NameError: Please run contrib/MADDPG instead
is included three times, I assume it's related to this.
2019-12-14 17:57:42,143 INFO resource_spec.py:205 -- Starting Ray with 39.11 GiB memory available for workers and up to 2.79 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
2019-12-14 17:57:42,516 WARNING sample.py:31 -- DeprecationWarning: wrapping <function main.<locals>.<lambda> at 0x7f115303cd90> with tune.function() is no longer needed
2019-12-14 17:57:42,599 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'numpy.dtype'> by using pickle. This may be inefficient.
2019-12-14 17:57:42,608 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'mtrand.RandomState'> by using pickle. This may be inefficient.
(pid=12579) 2019-12-14 17:57:44,655 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 17:57:45,381 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray_MADDPG:train() (pid=12579, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
Trainer.__init__(self, config, env, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 492, in _setup
self._init(self.config, self.env_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 109, in _init
self.config["num_workers"])
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 537, in _make_workers
logdir=self.logdir)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 64, in __init__
RolloutWorker, env_creator, policy, 0, self._local_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 220, in _make_worker
_fake_sampler=config.get("_fake_sampler", False))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 348, in __init__
self._build_policy_map(policy_dict, policy_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 764, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 158, in __init__
scope="actor"))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 368, in _build_actor_network
sampler = tfp.distributions.RelaxedOneHotCategorical(
AttributeError: 'NoneType' object has no attribute 'distributions'
2019-12-14 17:57:45,386 INFO trial_runner.py:615 -- Attempting to recover trial state from last checkpoint.
2019-12-14 17:57:46,464 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(NameError): ray__SeeContrib:train() (pid=12582, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/__init__.py", line 58, in _setup
"Please run `contrib/{}` instead.".format(name))
**NameError: Please run `contrib/MADDPG` instead.**
2019-12-14 17:57:46,468 INFO trial_runner.py:615 -- Attempting to recover trial state from last checkpoint.
(pid=12582) 2019-12-14 17:57:46,461 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 17:57:47,576 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(NameError): ray__SeeContrib:train() (pid=12576, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/__init__.py", line 58, in _setup
"Please run `contrib/{}` instead.".format(name))
**NameError: Please run `contrib/MADDPG` instead.**
2019-12-14 17:57:47,579 INFO trial_runner.py:615 -- Attempting to recover trial state from last checkpoint.
(pid=12576) 2019-12-14 17:57:47,572 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 17:57:48,644 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(NameError): ray__SeeContrib:train() (pid=12578, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/__init__.py", line 58, in _setup
"Please run `contrib/{}` instead.".format(name))
**NameError: Please run `contrib/MADDPG` instead.**
Traceback (most recent call last):
File "run_maddpg.py", line 182, in <module>
main(args)
File "run_maddpg.py", line 177, in main
}, verbose=0)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/tune.py", line 334, in run_experiments
return_trials=True)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/tune.py", line 269, in run
raise TuneError("Trials did not complete", errored_trials)
ray.tune.error.TuneError: ('Trials did not complete', [MADDPG_mpe_0])
(pid=12578) 2019-12-14 17:57:48,639 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
Ahh sorry, you also need to change:
run_experiments({
"MADDPG_RLLib": {
"run": "contrib/MADDPG",
},
I've updated the PR but can't test this right now unfortunately, so let me know if there are still problems. I was trying to help out with this issue: https://github.com/ray-project/ray/issues/6476
I got to this PR from having the same error as in that bug actually. Thanks the help with this too. There are still errors with that fix:
justinkterry@Prophet:~/Documents/maddpg-rllib$ python3 run_maddpg.py
2019-12-14 18:33:21,718 INFO resource_spec.py:205 -- Starting Ray with 38.43 GiB memory available for workers and up to 2.79 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
2019-12-14 18:33:22,053 WARNING sample.py:31 -- DeprecationWarning: wrapping <function main.<locals>.<lambda> at 0x7fdeb389ad90> with tune.function() is no longer needed
2019-12-14 18:33:22,127 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'numpy.dtype'> by using pickle. This may be inefficient.
2019-12-14 18:33:22,136 WARNING worker.py:348 -- WARNING: Falling back to serializing objects of type <class 'mtrand.RandomState'> by using pickle. This may be inefficient.
(pid=14615) 2019-12-14 18:33:24,168 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 18:33:24,860 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray_MADDPG:train() (pid=14615, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
Trainer.__init__(self, config, env, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 492, in _setup
self._init(self.config, self.env_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 109, in _init
self.config["num_workers"])
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 537, in _make_workers
logdir=self.logdir)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 64, in __init__
RolloutWorker, env_creator, policy, 0, self._local_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 220, in _make_worker
_fake_sampler=config.get("_fake_sampler", False))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 348, in __init__
self._build_policy_map(policy_dict, policy_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 764, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 158, in __init__
scope="actor"))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 368, in _build_actor_network
sampler = tfp.distributions.RelaxedOneHotCategorical(
AttributeError: 'NoneType' object has no attribute 'distributions'
2019-12-14 18:33:24,865 INFO trial_runner.py:615 -- Attempting to recover trial state from last checkpoint.
(pid=14604) 2019-12-14 18:33:25,922 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 18:33:26,632 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray_MADDPG:train() (pid=14604, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
Trainer.__init__(self, config, env, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 492, in _setup
self._init(self.config, self.env_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 109, in _init
self.config["num_workers"])
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 537, in _make_workers
logdir=self.logdir)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 64, in __init__
RolloutWorker, env_creator, policy, 0, self._local_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 220, in _make_worker
_fake_sampler=config.get("_fake_sampler", False))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 348, in __init__
self._build_policy_map(policy_dict, policy_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 764, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 158, in __init__
scope="actor"))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 368, in _build_actor_network
sampler = tfp.distributions.RelaxedOneHotCategorical(
AttributeError: 'NoneType' object has no attribute 'distributions'
2019-12-14 18:33:26,637 INFO trial_runner.py:615 -- Attempting to recover trial state from last checkpoint.
(pid=14605) 2019-12-14 18:33:27,692 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 18:33:28,390 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray_MADDPG:train() (pid=14605, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
Trainer.__init__(self, config, env, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 492, in _setup
self._init(self.config, self.env_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 109, in _init
self.config["num_workers"])
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 537, in _make_workers
logdir=self.logdir)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 64, in __init__
RolloutWorker, env_creator, policy, 0, self._local_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 220, in _make_worker
_fake_sampler=config.get("_fake_sampler", False))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 348, in __init__
self._build_policy_map(policy_dict, policy_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 764, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 158, in __init__
scope="actor"))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 368, in _build_actor_network
sampler = tfp.distributions.RelaxedOneHotCategorical(
AttributeError: 'NoneType' object has no attribute 'distributions'
2019-12-14 18:33:28,394 INFO trial_runner.py:615 -- Attempting to recover trial state from last checkpoint.
(pid=14603) 2019-12-14 18:33:29,456 INFO trainer.py:345 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2019-12-14 18:33:30,136 ERROR trial_runner.py:569 -- Error processing event.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trial_runner.py", line 515, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/ray_trial_executor.py", line 351, in fetch_result
result = ray.get(trial_future[0])
File "/usr/local/lib/python3.7/dist-packages/ray/worker.py", line 2121, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray_MADDPG:train() (pid=14603, host=Prophet)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
Trainer.__init__(self, config, env, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 372, in __init__
Trainable.__init__(self, config, logger_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/trainable.py", line 96, in __init__
self._setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 492, in _setup
self._init(self.config, self.env_creator)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer_template.py", line 109, in _init
self.config["num_workers"])
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/agents/trainer.py", line 537, in _make_workers
logdir=self.logdir)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 64, in __init__
RolloutWorker, env_creator, policy, 0, self._local_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/worker_set.py", line 220, in _make_worker
_fake_sampler=config.get("_fake_sampler", False))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 348, in __init__
self._build_policy_map(policy_dict, policy_config)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 764, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 158, in __init__
scope="actor"))
File "/usr/local/lib/python3.7/dist-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 368, in _build_actor_network
sampler = tfp.distributions.RelaxedOneHotCategorical(
AttributeError: 'NoneType' object has no attribute 'distributions'
Traceback (most recent call last):
File "run_maddpg.py", line 182, in <module>
main(args)
File "run_maddpg.py", line 177, in main
}, verbose=0)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/tune.py", line 334, in run_experiments
return_trials=True)
File "/usr/local/lib/python3.7/dist-packages/ray/tune/tune.py", line 269, in run
raise TuneError("Trials did not complete", errored_trials)
ray.tune.error.TuneError: ('Trials did not complete', [contrib_MADDPG_mpe_0])
tfp is Tensorflow Probability which I think you just need to pip install.
https://www.tensorflow.org/probability
Hopefully that works. Also, if you are able to benchmark this code it would be great. This project is unrelated to me, but in my brief tests I experienced some training issues when using Tensoflow 2.0.
Thank you. I'm using tensorflow v1.14 at the moment, because stable baselines is not compatible with tf2. I get the exact same error after installing tensorflow probability. Here's all my packages for reference:
justinkterry@Prophet:~/Documents/maddpg-rllib$ pip3 list
Package Version Location
----------------------- ----------- ---------------------------------------------
absl-py 0.8.1
altair 3.2.0
apturl 0.5.2
asn1crypto 0.24.0
astor 0.8.0
atari-py 0.2.6
attrs 19.3.0
backcall 0.1.0
bleach 3.1.0
blinker 1.4
box2d-py 2.3.8
Brlapi 0.6.7
certifi 2019.9.11
cffi 1.12.3
chardet 3.0.4
chrome-gnome-shell 0.0.0
Click 7.0
cloudpickle 1.1.1
colorama 0.4.3
colour 0.1.5
command-not-found 0.3
cryptography 2.3
cupshelpers 1.0
cycler 0.10.0
dash 1.4.1
dash-core-components 1.3.1
dash-html-components 1.0.1
dash-renderer 1.1.2
dash-table 4.4.1
decorator 4.4.0
defer 1.0.6
defusedxml 0.6.0
distro 1.3.0
distro-info 0.21ubuntu2
entrypoints 0.3
enum34 1.1.6
filelock 3.0.12
flake8 3.7.8
Flask 1.1.1
Flask-Compress 1.4.0
funcsigs 1.0.2
future 0.18.1
gast 0.2.2
google-pasta 0.1.8
GPy 1.9.8
grpcio 1.24.3
gym 0.15.3
h5py 2.10.0
httplib2 0.11.3
idna 2.8
imageio 2.5.0
importlib-metadata 0.23
ipykernel 5.1.2
ipython 7.8.0
ipython-genutils 0.2.0
ipywidgets 7.5.1
itsdangerous 1.1.0
jedi 0.15.1
Jinja2 2.10.1
joblib 0.14.0
jsonschema 3.1.1
jupyter 1.0.0
jupyter-client 5.3.3
jupyter-console 6.0.0
jupyter-core 4.6.0
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.0
keyboard 0.13.4
keyring 17.1.1
keyrings.alt 3.1.1
kiwisolver 1.1.0
language-selector 0.1
launchpadlib 1.10.6
lazr.restfulclient 0.14.2
lazr.uri 1.0.3
louis 3.8.0
lz4 2.2.1
macaroonbakery 1.2.1
Markdown 3.1.1
MarkupSafe 1.1.1
matplotlib 3.1.1
mccabe 0.6.1
mistune 0.8.4
mock 3.0.5
more-itertools 7.2.0
mpi4py 3.0.3
multiagent 0.0.1 /home/justinkterry/Documents/maddpg-rllib/MPE
nbconvert 5.6.0
nbformat 4.4.0
netifaces 0.10.4
networkx 2.3
notebook 6.0.1
numexpr 2.7.0
numpy 1.16.2
numpy-stl 2.10.1
oauthlib 2.1.0
olefile 0.46
opencv-python 4.1.1.26
packaging 19.2
pandas 0.25.1
pandocfilters 1.4.2
paramz 0.9.5
parso 0.5.1
pexpect 4.6.0
pickleshare 0.7.5
Pillow 5.4.1
Pillow-SIMD 6.0.0.post0
pip 18.1
plotly 4.2.0
pluggy 0.13.1
prometheus-client 0.7.1
prompt-toolkit 2.0.9
protobuf 3.11.1
psutil 5.6.7
psycopg2 2.8.3
ptyprocess 0.6.0
py 1.8.0
pycairo 1.16.2
pycodestyle 2.5.0
pycparser 2.19
pycrypto 2.6.1
pycups 1.9.73
pyflakes 2.1.1
pygame 1.9.6
pyglet 1.3.2
Pygments 2.4.2
PyGObject 3.32.0
PyJWT 1.7.0
pymacaroons 0.13.0
pymunk 5.5.0
PyNaCl 1.3.0
pynput 1.4.5
pyparsing 2.4.2
pyRFC3339 1.1
pyrsistent 0.15.4
pytest 5.3.1
python-apt 1.8.4
python-dateutil 2.7.3
python-debian 0.1.34
python-utils 2.3.0
python-xlib 0.25
pytz 2019.3
PyWavelets 1.0.3
pyxdg 0.25
PyYAML 3.13
pyzmq 18.1.0
qtconsole 4.5.5
ray 0.7.6
redis 3.3.11
reportlab 3.5.18
requests 2.22.0
requests-unixsocket 0.1.5
retrying 1.3.3
scikit-image 0.15.0
scikit-learn 0.21.3
scipy 1.2.1
screen-resolution-extra 0.0.0
seaborn 0.9.0
SecretStorage 2.3.1
selenium 3.141.0
Send2Trash 1.5.0
setproctitle 1.1.10
setuptools 41.4.0
simplejson 3.16.0
six 1.12.0
sklearn 0.0
stable-baselines 2.8.0
system-service 0.3
systemd-python 234
tables 3.5.2
tensorboard 1.14.0
tensorflow 1.14.0
tensorflow-estimator 1.14.0
tensorflow-probability 0.8.0
termcolor 1.1.0
terminado 0.8.2
testpath 0.4.2
toolz 0.10.0
torch 1.2.0
torchvision 0.4.0
tornado 6.0.3
traitlets 4.3.3
ubuntu-advantage-tools 19.2
ubuntu-drivers-common 0.0.0
ufw 0.36
unattended-upgrades 0.1
urllib3 1.25.6
wadllib 1.3.3
wcwidth 0.1.7
webencodings 0.5.1
Werkzeug 0.16.0
wheel 0.32.3
widgetsnbextension 3.5.1
wrapt 1.11.2
xkit 0.0.0
zipp 0.6.0
@justinkterry have you try to install tensorflow-probability==0.7.0
which tensorflow-probability version 0.7.0?
That fixed it. That needs to be included in the documentation somewhere.
On Mon, Dec 16, 2019 at 11:54 AM dragon28 [email protected] wrote:
@justinkterry https://github.com/justinkterry have you try to install tensorflow-probability==0.7.0 which tensorflow-probability version 0.7.0?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wsjeon/maddpg-rllib/pull/7?email_source=notifications&email_token=AEUF33AEJVYGXSHHAKBQX5TQY6XDHA5CNFSM4J2L2BUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG7LMDA#issuecomment-566146572, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEUF33GKGMHA2ENZYRL7IV3QY6XDHANCNFSM4J2L2BUA .
-- Thank you for your time, Justin Terry
@sanjeevanahilan @wsjeon @justinkterry
I think I found some errors related to memory:
$ python3 -Wall ./run_maddpg.py --scenario simple_tag
/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/usr/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
return f(*args, **kwds)
/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py:5506: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/dragon/.keras/keras.json' mode='r' encoding='UTF-8'>
_config = json.load(open(_config_path))
2020-01-04 18:19:05,520 INFO resource_spec.py:216 -- Starting Ray with 1.17 GiB memory available for workers and up to 0.93 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:593: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/redis-shard_0.err' mode='a' encoding='UTF-8'>
self.start_redis()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:593: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/redis-shard_0.out' mode='a' encoding='UTF-8'>
self.start_redis()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:593: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/redis.err' mode='a' encoding='UTF-8'>
self.start_redis()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:593: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/redis.out' mode='a' encoding='UTF-8'>
self.start_redis()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:594: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/monitor.out' mode='a' encoding='UTF-8'>
self.start_monitor()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:594: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/monitor.err' mode='a' encoding='UTF-8'>
self.start_monitor()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:595: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/raylet_monitor.out' mode='a' encoding='UTF-8'>
self.start_raylet_monitor()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:595: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/raylet_monitor.err' mode='a' encoding='UTF-8'>
self.start_raylet_monitor()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:606: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/plasma_store.out' mode='a' encoding='UTF-8'>
self.start_plasma_store()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:606: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/plasma_store.err' mode='a' encoding='UTF-8'>
self.start_plasma_store()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:607: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/raylet.out' mode='a' encoding='UTF-8'>
self.start_raylet()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:607: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/raylet.err' mode='a' encoding='UTF-8'>
self.start_raylet()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:609: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/reporter.out' mode='a' encoding='UTF-8'>
self.start_reporter()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:609: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/reporter.err' mode='a' encoding='UTF-8'>
self.start_reporter()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:612: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/log_monitor.out' mode='a' encoding='UTF-8'>
self.start_log_monitor()
/home/dragon/.local/lib/python3.6/site-packages/ray/node.py:612: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/ray/session_2020-01-04_18-19-05_509313_31535/logs/log_monitor.err' mode='a' encoding='UTF-8'>
self.start_log_monitor()
2020-01-04 18:19:05,925 WARNING sample.py:31 -- DeprecationWarning: wrapping <function main.<locals>.<lambda> at 0x7f246a0c9950> with tune.function() is no longer needed
(pid=31584) 2020-01-04 18:19:08,573 INFO trainer.py:371 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
(pid=31584) 2020-01-04 18:19:08,575 INFO trainer.py:512 -- Current log_level is ERROR. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
(pid=31584) 2020-01-04 18:19:13,723 WARNING util.py:45 -- Install gputil for GPU system monitoring.
(pid=31584) /home/dragon/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py:2957: RuntimeWarning: Mean of empty slice.
(pid=31584) out=out, **kwargs)
(pid=31584) /home/dragon/.local/lib/python3.6/site-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
(pid=31584) ret = ret.dtype.type(ret / rcount)
2020-01-04 19:22:31,599 ERROR trial_runner.py:476 -- Error processing event.
Traceback (most recent call last):
File "/home/dragon/.local/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 420, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 378, in fetch_result
result = ray.get(trial_future[0], DEFAULT_GET_TIMEOUT)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/worker.py", line 1456, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(RayOutOfMemoryError): ray::MADDPG.train() (pid=31584, ip=192.168.0.190)
File "python/ray/_raylet.pyx", line 636, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 619, in ray._raylet.execute_task.function_executor
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 444, in train
raise e
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 433, in train
result = Trainable.train(self)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/tune/trainable.py", line 176, in train
result = self._train()
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 138, in _train
res = collect_metrics_fn(self)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/contrib/maddpg/maddpg.py", line 170, in collect_metrics
result = trainer.collect_metrics()
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 753, in collect_metrics
selected_workers=selected_workers)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/optimizers/policy_optimizer.py", line 108, in collect_metrics
timeout_seconds=timeout_seconds)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/evaluation/metrics.py", line 75, in collect_episodes
metric_lists = ray_get_and_free(collected)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/rllib/utils/memory.py", line 33, in ray_get_and_free
result = ray.get(object_ids)
ray.exceptions.RayTaskError(RayOutOfMemoryError): ray::RolloutWorker (pid=31583, ip=192.168.0.190)
File "python/ray/_raylet.pyx", line 627, in ray._raylet.execute_task
File "/home/dragon/.local/lib/python3.6/site-packages/ray/memory_monitor.py", line 130, in raise_if_low_memory
self.error_threshold))
ray.memory_monitor.RayOutOfMemoryError: More than 95% of the memory on node WorkStation is used (14.83 / 15.61 GB). The top 10 memory consumers are:
PID MEM COMMAND
31584 7.03GiB ray::MADDPG.train()
5436 0.47GiB /usr/lib/firefox/firefox -new-window
30138 0.33GiB /usr/lib/x86_64-linux-gnu/opera/opera --type=renderer --field-trial-handle=3240866520022368210,60777
31583 0.33GiB ray::RolloutWorker
3178 0.29GiB /usr/bin/gnome-shell
25811 0.28GiB /usr/bin/python3 -m pyls --host 127.0.0.1 --port 2087 --tcp --check-parent-process
5482 0.24GiB /usr/lib/x86_64-linux-gnu/opera/opera --type=gpu-process --field-trial-handle=3240866520022368210,60
31535 0.24GiB python3 -Wall ./run_maddpg.py --scenario simple_tag
25753 0.2GiB /usr/bin/python3 /home/dragon/.local/bin/spyder3
5432 0.16GiB /usr/lib/x86_64-linux-gnu/opera/opera --new-window
In addition, up to 1.43 GiB of shared memory is currently being used by the Ray object store. You can set the object store size with the `object_store_memory` parameter when starting Ray, and the max Redis size with `redis_max_memory`. Note that Ray assumes all system memory is available for use by workers. If your system has other applications running, you should manually set these memory limits to a lower value.
Traceback (most recent call last):
File "./run_maddpg.py", line 187, in <module>
main(args)
File "./run_maddpg.py", line 182, in main
}, verbose=0)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/tune/tune.py", line 408, in run_experiments
return_trials=True)
File "/home/dragon/.local/lib/python3.6/site-packages/ray/tune/tune.py", line 327, in run
raise TuneError("Trials did not complete", errored_trials)
ray.tune.error.TuneError: ('Trials did not complete', [contrib_MADDPG_mpe_a339b59a])
OS: Ubuntu 18.04 64-bits
$ uname -a
Linux WorkStation 5.0.0-37-generic #40~18.04.1-Ubuntu SMP Thu Nov 14 12:06:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Python: Python version 3.6 (CPU only)
$ python3 --version
Python 3.6.9
Redis version 4.0.9
$ redis-server -v
Redis server v=4.0.9 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=9435c3c2879311f3
Tensorflow version 1.14.0
Tensorflow-estimator version 1.14.0
Tensorflow-metadata version 0.14.0
Tensorflow-probability version 0.7.0
RLLib version 0.8.0.dev7
Pip Details:
$ pip3 list
Package Version Location
----------------------------- ------------------- -------------------------------------------
absl-py 0.7.1
alabaster 0.7.12
apache-beam 2.16.0
appdirs 1.4.3
apturl 0.5.2
argh 0.26.2
asn1crypto 0.24.0
astor 0.8.0
astroid 2.2.5
asyncio 3.4.3
atari-py 0.2.6
atomicwrites 1.3.0
attrs 19.1.0
autopep8 1.4.4
avro-python3 1.9.0
Babel 2.6.0
backcall 0.1.0
backports.csv 1.0.7
baselines 0.1.6
beautifulsoup4 4.7.1
bintrees 2.0.7
bleach 3.1.0
blis 0.2.4
boto3 1.9.240
botocore 1.12.240
Box2D 2.3.2
Box2D-kengz 2.3.3
Brlapi 0.6.6
bs4 0.0.1
cachetools 3.1.1
certifi 2018.1.18
cffi 1.12.3
chardet 3.0.4
Click 7.0
cloudpickle 1.2.2
colorama 0.4.1
colour 0.1.5
command-not-found 0.3
contextlib2 0.5.5
crcmod 1.7
cryptography 2.1.4
cssselect 1.0.3
cupshelpers 1.0
cycler 0.10.0
cymem 2.0.2
Cython 0.29.10
data 0.4
dataclasses 0.7
decorator 4.4.0
defer 1.0.6
defusedxml 0.6.0
devscripts 2.17.12ubuntu1.1
diff-match-patch 20181111
dill 0.3.0
diskcache 3.1.1
distro-info 0.18ubuntu0.18.04.1
dlib 19.17.99
dm-sonnet 1.34
docopt 0.6.2
docutils 0.14
dopamine-rl 2.0.5 /home/dragon/machine_learning/dopamine
easydict 1.9
editdistance 0.5.3
entrypoints 0.3
enum34 1.1.6
et-xmlfile 1.0.1
fake-useragent 0.1.11
fastavro 0.21.24
fasteners 0.15
filelock 3.0.12
findspark 1.3.0
flake8 3.7.9
flatbuffers 1.10
funcsigs 1.0.2
future 0.17.1
gast 0.2.2
gin-config 0.1.4
gitdb2 2.0.5
GitPython 2.1.11
glfw 1.8.1
glob2 0.6
gluonts 0.3.3
google-api-core 1.13.0
google-api-python-client 1.7.11
google-apitools 0.5.28
google-auth 1.6.3
google-auth-httplib2 0.0.3
google-auth-oauthlib 0.4.1
google-cloud-bigquery 1.6.1
google-cloud-bigtable 0.32.2
google-cloud-core 0.29.1
google-cloud-datastore 1.7.4
google-cloud-pubsub 0.39.1
google-pasta 0.1.7
google-resumable-media 0.3.2
googleapis-common-protos 1.6.0
gpg 1.10.0
graphviz 0.8.4
grpc-google-iam-v1 0.11.4
grpcio 1.20.1
gym 0.15.3
gym-ctc-executioner 0.0.1
gym-ctc-marketmaker 0.0.1
gym-test 0.0.1 /home/dragon/reinforcementlearning/gym-test
h5py 2.9.0
handyspark 0.2.2a1
hdfs 2.5.8
helpdev 0.6.10
holidays 0.9.11
httplib2 0.9.2
icc-rt 2019.0
idna 2.6
imageio 2.5.0
imagesize 1.1.0
importlib-metadata 0.17
imutils 0.5.3
intel-openmp 2019.0
intervaltree 3.0.2
ipykernel 5.1.3
ipython 7.5.0
ipython-genutils 0.2.0
ipywidgets 7.4.2
isort 4.3.20
jdcal 1.4.1
jedi 0.14.1
Jinja2 2.10.1
jmespath 0.9.4
joblib 0.13.2
jsonschema 3.0.1
jupyter 1.0.0
jupyter-client 5.3.4
jupyter-console 6.0.0
jupyter-core 4.6.1
Keras 2.3.1
Keras-Applications 1.0.8
Keras-Preprocessing 1.0.9
keras-rl 0.4.2
keyring 10.6.0
keyrings.alt 3.0
kiwisolver 1.1.0
language-selector 0.1
latex 0.7.0
launchpadlib 1.10.6
lazr.restfulclient 0.13.5
lazr.uri 1.0.3
lazy-object-proxy 1.4.1
lockfile 0.12.2
louis 3.5.0
lxml 4.3.3
lz4 2.2.1
macaroonbakery 1.1.3
maddpg 0.0.1
Mako 1.0.7
Markdown 3.1.1
MarkupSafe 1.0
matplotlib 3.1.0
mccabe 0.6.1
meld 3.18.0
mistune 0.8.4
mkl 2019.0
mkl-fft 1.0.6
mkl-random 1.0.1.1
mock 2.0.0
monotonic 1.5
more-itertools 7.0.0
mpi4py 3.1.0a0
mpmath 1.1.0
multiagent 0.0.1
murmurhash 1.0.2
mxnet 1.4.1
nbconvert 5.5.0
nbformat 4.4.0
neat-python 0.92
netifaces 0.10.4
networkx 2.3
nltk 3.4.5
notebook 5.7.8
numexpr 2.6.9
numpy 1.14.6
numpy-stl 2.10.1
numpydoc 0.9.1
oauth 1.0.1
oauth2client 3.0.0
oauthlib 3.1.0
odfpy 1.4.0
olefile 0.45.1
opencv-contrib-python 4.1.1.26
opencv-python 4.1.1.26
opencv-python-headless 4.1.2.30
openpyxl 2.4.11
opt-einsum 3.1.0
ordered-set 3.1.1
packaging 19.0
pandas 0.25.2
pandocfilters 1.4.2
parse 1.12.0
parso 0.5.1
pathtools 0.1.2
pbr 5.4.0
pexpect 4.7.0
pickleshare 0.7.5
Pillow 6.1.0
pip 19.3.1
plac 0.9.6
plotly 3.9.0
pluggy 0.12.0
preshed 2.0.1
prometheus-client 0.6.0
prompt-toolkit 2.0.9
protobuf 3.11.0
psutil 5.6.2
ptyprocess 0.6.0
py 1.8.0
py-spy 0.3.1
py4j 0.10.7
pyaes 1.6.1
pyarrow 0.14.1
pyasn1 0.4.5
pyasn1-modules 0.2.5
pycairo 1.16.2
pyclipper 1.1.0.post1
pycodestyle 2.5.0
pycparser 2.19
pycrypto 2.6.1
pycups 1.9.73
pydantic 0.28
pydocstyle 5.0.1
pydot 1.2.4
pydotplus 2.0.2
PyDrive 1.3.1
pyee 6.0.0
pyflakes 2.1.1
pygame 1.9.6
pyglet 1.3.2
Pygments 2.4.0
pygobject 3.26.1
PyLaTeX 1.3.1
pylint 2.3.1
pymacaroons 0.13.0
pymongo 3.9.0
Pympler 0.8
PyNaCl 1.1.2
PyOpenGL 3.1.0
PyOpenGL-accelerate 3.1.0
pyparsing 2.4.0
pyppeteer 0.0.25
pyprob 0.13.0
PyQt5 5.12.2
PyQt5-sip 4.19.17
PyQtWebEngine 5.12.1
pyquery 1.4.0
pyRFC3339 1.0
pyrsistent 0.15.2
pyspark 2.4.3
pytesseract 0.3.0
pytest 4.6.0
pytexit 0.3.3
python-apt 1.6.4
python-dateutil 2.8.0
python-debian 0.1.32
python-jsonrpc-server 0.3.2
python-language-server 0.31.2
python-magic 0.4.16
python-telegram 0.10.0
python-utils 2.3.0
pytz 2018.3
PyWavelets 1.0.3
pyxdg 0.26
PyYAML 3.12
pyzmq 18.0.1
QDarkStyle 2.7
QtAwesome 0.5.7
qtconsole 4.6.0
QtPy 1.7.1
ray 0.8.0.dev7
records 0.5.3
redis 3.3.11
regex 2019.8.19
reportlab 3.4.0
requests 2.22.0
requests-html 0.10.0
requests-oauthlib 1.3.0
requests-unixsocket 0.1.5
retrying 1.3.3
rl 3.0
rope 0.14.0
rsa 4.0
s3transfer 0.2.1
sacremoses 0.0.35
scikit-image 0.15.0
scikit-learn 0.21.1
scipy 1.3.0
scons 3.0.5
screen-resolution-extra 0.0.0
seaborn 0.9.0
SecretStorage 2.3.1
selenium 3.141.0
semantic-version 2.6.0
Send2Trash 1.5.0
sentencepiece 0.1.83
setproctitle 1.1.10
setuptools 41.0.1
shutilwhich 1.1.0
simplejson 3.13.2
six 1.13.0
smmap2 2.0.5
snowballstemmer 1.2.1
sortedcontainers 2.1.0
soupsieve 1.9.1
spacy 2.1.4
Sphinx 2.0.1
sphinxcontrib-applehelp 1.0.1
sphinxcontrib-devhelp 1.0.1
sphinxcontrib-htmlhelp 1.0.2
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.2
sphinxcontrib-serializinghtml 1.1.3
spyder 4.0.0
spyder-kernels 1.8.1
SQLAlchemy 1.3.8
srsly 0.0.7
stable-baselines 2.8.0
sympy 1.4
system-service 0.3
systemd-python 234
tables 3.5.2
tablib 0.13.0
tabulate 0.8.3
tbb 2019.0
tbb4py 2019.0
Telethon 1.10.6
tempdir 0.7.1
tensorboard 1.14.0
tensorflow 1.14.0
tensorflow-estimator 1.14.0
tensorflow-metadata 0.14.0
tensorflow-probability 0.7.0
tensorflow-serving-api 1.14.0
tensorflow-transform 0.14.0
termcolor 1.1.0
terminado 0.8.2
testpath 0.4.2
testresources 2.0.1
tf-agents 0.2.0rc2
tfx-bsl 0.15.0
thinc 7.0.4
torch 1.2.0+cpu
torchvision 0.4.0+cpu
tornado 6.0.2
tqdm 4.32.1
trading-env 0.0.1.dev0
traitlets 4.3.2
transformers 2.1.0
trfl 1.0.1
typed-ast 1.3.5
ubuntu-drivers-common 0.0.0
ufw 0.36
ujson 1.35
unattended-upgrades 0.1
unicodecsv 0.14.1
unidiff 0.5.4
uritemplate 3.0.0
urllib3 1.22
usb-creator 0.3.3
virtualenv 16.6.0
w3lib 1.20.0
wadllib 1.3.2
wasabi 0.2.2
watchdog 0.9.0
wcwidth 0.1.7
webencodings 0.5.1
websockets 7.0
Werkzeug 0.15.4
wget 3.2
wheel 0.30.0
widgetsnbextension 3.4.2
wrapt 1.11.1
wurlitzer 2.0.0
xkit 0.0.0
xlrd 1.2.0
xlwt 1.3.0
yapf 0.29.0
zipp 0.5.1
zmq 0.0.0
zope.interface 4.3.2
On the other hand, it also unable to execute simple_world_comm scenario
python3 -Wall ./run_maddpg.py --scenario simple_world_comm
/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/usr/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
return f(*args, **kwds)
/home/dragon/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py:5506: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/dragon/.keras/keras.json' mode='r' encoding='UTF-8'>
_config = json.load(open(_config_path))
usage: MADDPG with OpenAI MPE [-h]
[--scenario {simple,simple_speaker_listener,simple_crypto,simple_push,simple_tag,simple_spread,simple_adversary}]
[--max-episode-len MAX_EPISODE_LEN]
[--num-episodes NUM_EPISODES]
[--num-adversaries NUM_ADVERSARIES]
[--good-policy GOOD_POLICY]
[--adv-policy ADV_POLICY] [--lr LR]
[--gamma GAMMA]
[--sample-batch-size SAMPLE_BATCH_SIZE]
[--train-batch-size TRAIN_BATCH_SIZE]
[--n-step N_STEP] [--num-units NUM_UNITS]
[--checkpoint-freq CHECKPOINT_FREQ]
[--local-dir LOCAL_DIR] [--restore RESTORE]
[--num-workers NUM_WORKERS]
[--num-envs-per-worker NUM_ENVS_PER_WORKER]
[--num-gpus NUM_GPUS]
MADDPG with OpenAI MPE: error: argument --scenario: invalid choice: 'simple_world_comm' (choose from 'simple', 'simple_speaker_listener', 'simple_crypto', 'simple_push', 'simple_tag', 'simple_spread', 'simple_adversary')
@dragon28 the real world com scenario isn't one of the currently supported MPE games (read ~line 34 of run_maddpg.py where it parses the --scenario arg).
In terms of the memory error, it's an issue with ray configuration instead of this code. You have to edit the config for how much ram workers, etc. can use in ray (google it). Ray infers defaults based on the system, so my 64GB RAM workstation can run it fine without changes, but my 16GB RAM laptop can't.
This default config on my workstation works, for reference:
2020-01-04 17:34:56,871 INFO resource_spec.py:216 -- Starting Ray with 39.06 GiB memory available for workers and up to 2.79 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
@dragon28
Hey, I'm having the exact same memory issue as you very reproducibly. There appears to be a leak in the number of workers? I'll update this when I figured out what's going on more exactly.
Hey all, this repo isn't being maintained, so I forked it, made these fixes, and the maintainers of RLlib/ray made it the new official example repo of the maddpg example code. See https://github.com/ray-project/ray/pull/6831 and https://github.com/justinkterry/maddpg-rllib.
hey, @justinkterry I got the same error as in that bug actually. Thanks the help with this too. There are still errors with that fix:
2020-06-04 15:16:12,567 ERROR trial_runner.py:519 -- Trial MADDPG_mpe_420fc_00000: Error processing event.
Traceback (most recent call last):
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 467, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 430, in fetch_result
result = ray.get(trial_future[0], DEFAULT_GET_TIMEOUT)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/worker.py", line 1474, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::MADDPG.train() (pid=148180, ip=192.168.1.106)
File "python/ray/_raylet.pyx", line 407, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 442, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 445, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 446, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 400, in ray._raylet.execute_task.function_executor
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 90, in init
Trainer.init(self, config, env, logger_creator)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 452, in init
super().init(config, logger_creator)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/trainable.py", line 174, in init
self._setup(copy.deepcopy(self.config))
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 627, in _setup
self._init(self.config, self.env_creator)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 115, in _init
self.config["num_workers"])
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 700, in _make_workers
logdir=self.logdir)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 59, in init
RolloutWorker, env_creator, policy, 0, self._local_config)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 282, in _make_worker
extra_python_environs=extra_python_environs)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 378, in init
self._build_policy_map(policy_dict, policy_config)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 930, in _build_policy_map
policy_map[name] = cls(obs_space, act_space, merged_conf)
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 155, in init
scope="actor"))
File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 371, in _build_actor_network
sampler = tfp.distributions.RelaxedOneHotCategorical(
AttributeError: 'NoneType' object has no attribute 'distributions'
Traceback (most recent call last):
File "run_maddpg.py", line 183, in
You have to use my fork
On Thu, Jun 4, 2020 at 3:17 AM Guoth [email protected] wrote:
hey, @justinkterry https://github.com/justinkterry I got the same error as in that bug actually. Thanks the help with this too. There are still errors with that fix:
2020-06-04 15:16:12,567 ERROR trial_runner.py:519 -- Trial MADDPG_mpe_420fc_00000: Error processing event. Traceback (most recent call last): File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 467, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 430, in fetch_result result = ray.get(trial_future[0], DEFAULT_GET_TIMEOUT) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/worker.py", line 1474, in get raise value.as_instanceof_cause() ray.exceptions.RayTaskError(AttributeError): ray::MADDPG.train() (pid=148180, ip=192.168.1.106) File "python/ray/_raylet.pyx", line 407, in ray._raylet.execute_task File "python/ray/_raylet.pyx", line 442, in ray._raylet.execute_task File "python/ray/_raylet.pyx", line 445, in ray._raylet.execute_task File "python/ray/_raylet.pyx", line 446, in ray._raylet.execute_task File "python/ray/_raylet.pyx", line 400, in ray._raylet.execute_task.function_executor File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 90, in init Trainer.init(self, config, env, logger_creator) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 452, in init super().init(config, logger_creator) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/trainable.py", line 174, in init self._setup(copy.deepcopy(self.config)) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 627, in _setup self._init(self.config, self.env_creator) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 115, in _init self.config["num_workers"]) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 700, in _make_workers logdir=self.logdir) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 59, in init RolloutWorker, env_creator, policy, 0, self._local_config) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 282, in _make_worker extra_python_environs=extra_python_environs) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 378, in init self._build_policy_map(policy_dict, policy_config) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 930, in _build_policy_map policy_map[name] = cls(obs_space, act_space, merged_conf) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 155, in init scope="actor")) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/rllib/contrib/maddpg/maddpg_policy.py", line 371, in _build_actor_network sampler = tfp.distributions.RelaxedOneHotCategorical( AttributeError: 'NoneType' object has no attribute 'distributions' Traceback (most recent call last): File "run_maddpg.py", line 183, in main(args) File "run_maddpg.py", line 178, in main }, verbose=0) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/tune.py", line 411, in run_experiments return_trials=True) File "/home/aics/anaconda3/envs/tf/lib/python3.6/site-packages/ray/tune/tune.py", line 347, in run raise TuneError("Trials did not complete", incomplete_trials) ray.tune.error.TuneError: ('Trials did not complete', [MADDPG_mpe_420fc_00000]) (tf) aics@aics-Super-Server:~/RayRLlib/maddpg-rllib$
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wsjeon/maddpg-rllib/pull/7#issuecomment-638653788, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEUF33GDHJAFSUGNECOIOM3RU5DBPANCNFSM4J2L2BUA .
-- Thank you for your time, Justin Terry