Multiagent_merge fails

[SHITIANYU-hue]

Bug Description

Traceback (most recent call last): File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 498, in _process_trial result = self.trial_executor.fetch_result(trial) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 342, in fetch_result result = ray.get(trial_future[0]) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/worker.py", line 2247, in get raise value ray.exceptions.RayTaskError: ^[[36mray_PPO:train()^[[39m (pid=20305, host=153SMART-08-ULTS) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 369, in train raise e File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 358, in train result = Trainable.train(self) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trainable.py", line 171, in train result = self._train() File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 126, in _train fetches = self.optimizer.step() File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/multi_gpu_optimizer.py", line 140, in step self.num_envs_per_worker, self.train_batch_size) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/rollout.py", line 29, in collect_samples next_sample = ray_get_and_free(fut_sample) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/utils/memory.py", line 33, in ray_get_and_free result = ray.get(object_ids) ray.exceptions.RayTaskError: ^[[36mray_RolloutWorker:sample()^[[39m (pid=20304, host=153SMART-08-ULTS) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 453, in sample batches = [self.input_reader.next()] File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 56, in next batches = [self.get_data()] File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 97, in get_data item = next(self.rollout_provider) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 313, in _env_runner soft_horizon) File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 451, in _process_observations episode.batch_builder.check_missing_dones() File "/home/tianyushi/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sample_batch_builder.py", line 187, in check_missing_dones "Please ensure that you include the last observations " ValueError: The environment terminated for all agents, but we still don't have a last observation for agent flow_10.18 (policy av). Please ensure that you include the last observations of all live agents when setting 'all' done to True.

Bug Reproduce

File in which the bug occurred: file.py

The exact command that you did run that caused the bug: python train.py multiagent_merge

Output Log

Full error output log:

error log

Screenshots

Screenshots:

Versions

• OS: ? (For example, RedHat, macOS Mojave, ...]
• Flow version: ? (run python -c "import flow; print(flow.__version__)")
• Flow commit number: ? (run git log and copy the commit number)