flow
flow copied to clipboard
Published
20 hours ago •
flow-project
flow-project
singleagent_merge training fails
Bug Description
When I train singleagent_merge, it fails
Bug Reproduce
python train.py singleagent_merge
Output Log
Traceback (most recent call last):
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 498, in _process_trial
result = self.trial_executor.fetch_result(trial)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 342, in fetch_result
result = ray.get(trial_future[0])
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/worker.py", line 2247, in get
raise value
ray.exceptions.RayTaskError: [36mray_PPO:train()[39m (pid=14406, host=intel-pc)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 369, in train
raise e
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 358, in train
result = Trainable.train(self)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/tune/trainable.py", line 171, in train
result = self._train()
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 126, in _train
fetches = self.optimizer.step()
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/multi_gpu_optimizer.py", line 140, in step
self.num_envs_per_worker, self.train_batch_size)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/optimizers/rollout.py", line 29, in collect_samples
next_sample = ray_get_and_free(fut_sample)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/utils/memory.py", line 33, in ray_get_and_free
result = ray.get(object_ids)
ray.exceptions.RayTaskError: [36mray_RolloutWorker:sample()[39m (pid=14402, host=intel-pc)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 453, in sample
batches = [self.input_reader.next()]
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 56, in next
batches = [self.get_data()]
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 97, in get_data
item = next(self.rollout_provider)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 313, in _env_runner
soft_horizon)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/evaluation/sampler.py", line 401, in _process_observations
policy_id).transform(raw_obs)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/models/preprocessors.py", line 166, in transform
self.check_shape(observation)
File "/home/intel/anaconda3/envs/flow/lib/python3.6/site-packages/ray/rllib/models/preprocessors.py", line 61, in check_shape
self._obs_space, observation)
ValueError: ('Observation outside expected value range', Box(280,), array([ 0.00000000e+00, 4.48674389e-03, 4.69197036e-02, 3.16010523e-05,
0.00000000e+00, 0.00000000e+00, 4.69065457e-02, 3.10963661e-02,
1.88532636e-01, 4.23515771e-01, 3.42134161e-02, 7.35134037e-02,
7.60705448e-06, 6.90067447e-03, 3.24158843e-01, 3.17126994e-01,
3.64110069e-01, 4.02902781e-01, 0.00000000e+00, 1.76898596e-03,
8.69775084e-03, 7.68905950e-03, 0.00000000e+00, 5.08732856e-04,
8.40520125e-02, 3.60284041e-02, 0.00000000e+00, 3.99014638e-01,
1.32984023e-01, 0.00000000e+00, 2.96668282e-02, 1.89124059e-03,
0.00000000e+00, 0.00000000e+00, 1.41107882e-05, 6.18246402e-05,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 3.24483209e-01, 4.01030390e-01,
4.71162745e-02, 4.70089797e-04, 0.00000000e+00, 2.85571169e-02,
2.87845099e-03, 2.83716878e-02, 2.21626102e-02, 5.26387533e-02,
0.00000000e+00, 0.00000000e+00, 2.28072147e-02, 5.68610825e-02,
3.24224414e-02, 6.54499515e-01, 9.06251461e-03, 4.06600015e-02,
6.74082556e-01, 6.99267902e-01, 5.60383461e-01, 6.35572339e-01,
1.13066070e-01, 2.47346859e-01, 2.77777281e-03, 2.79191731e-02,
3.33666667e-03, 3.07541214e-02, 3.33666667e-03, 2.83906531e-02,
1.30654777e-02, 4.92858732e-02, 7.32800789e-03, 2.16665080e-01,
6.54230012e-01, 0.00000000e+00, 3.33666667e-03, 3.23191679e-02,
2.11124944e-03, 2.71117168e-02, 1.26715522e-02, 3.76772208e-02,
1.26702344e-02, 3.76706168e-02, 2.51939438e-03, 2.75197527e-02,
0.00000000e+00, 0.00000000e+00, 7.60635793e-02, 8.34823607e-01,
2.28675005e-02, 5.33942033e-02, 7.26996557e-03, 3.29478805e-02,
1.62790698e-01, 1.62790698e-01, 4.65116279e-02, 4.65116279e-02,
0.00000000e+00, 0.00000000e+00, 3.48837209e-01, 3.48837209e-01,
2.09302326e-01, 2.09302326e-01, 9.30232558e-02, 9.30232558e-02,
2.79069767e-01, 2.79069767e-01, 3.95348837e-01, 3.95348837e-01,
2.55813953e-01, 2.55813953e-01, 1.39534884e-01, 1.39534884e-01,
3.25581395e-01, 3.25581395e-01, 4.41860465e-01, 4.41860465e-01,
4.88372093e-01, 4.88372093e-01, 3.72093023e-01, 3.72093023e-01,
5.58139535e-01, 0.00000000e+00, 6.74418605e-01, 6.74418605e-01,
5.34883721e-01, 5.34883721e-01, 4.18604651e-01, 4.18604651e-01,
6.04651163e-01, 6.04651163e-01, 7.20930233e-01, 7.20930233e-01,
0.00000000e+00, 0.00000000e+00, 4.65116279e-01, 4.65116279e-01,
6.51162791e-01, 6.51162791e-01, 7.67441860e-01, 7.67441860e-01,
2.16666667e-01, 5.00000000e-02, 5.00000000e-02, 1.50000000e-01,
2.00000000e-01, 1.16666667e-01, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 1.50000000e-01, 6.66666667e-02,
3.33333333e-02, 1.16666667e-01, 2.00000000e-01, 3.66666667e-01,
3.16666667e-01, 3.83333333e-01, 3.83333333e-01, 4.00000000e-01,
5.00000000e-02, 1.00000000e-01, 3.33333333e-02, 5.00000000e-02,
5.00000000e-02, 5.00000000e-02, 0.00000000e+00, 0.00000000e+00,
8.33333333e-02, 4.66666667e-01, 0.00000000e+00, 1.66666667e-02,
1.00000000e-01, 2.50000000e-01, 1.48685178e-01, 3.35974811e-01,
3.96160249e-01, 1.47097783e-01, 1.35245582e-01, 8.15259013e-02,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
7.66484153e-02, 2.23485215e-01, 3.20642919e-01, 2.02716680e-01,
8.04653056e-02, 5.25515315e-02, 1.09691206e-01, 1.10236476e-01,
1.01870119e-01, 4.05163967e-02, 2.68243040e-01, 1.20740103e-01,
3.62756800e-01, 4.28571429e-01, 1.12093114e-01, 4.19048617e-01,
0.00000000e+00, 0.00000000e+00, 7.41196468e-02, 7.77994982e-02,
0.00000000e+00, 1.32984023e-01, 7.19981731e-02, 6.87010532e-02,
3.00000000e+00, 2.00000000e+00, 1.00000000e+00, 0.00000000e+00,
2.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.00000000e+00,
0.00000000e+00, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.00000000e+00, 1.00000000e+00, 1.00000000e+00,
1.00000000e+00, 1.00000000e+00, 1.16638555e-01, 8.83361445e-01,
1.00000000e+00, -2.55643238e-02, 8.07981492e-04, 1.32984023e-01,
8.67015977e-01, 1.00000000e+00, 1.32984023e-01, 1.00000000e+00,
1.12093114e-01, 8.87906886e-01, 1.00000000e+00, 5.97181930e-02,
7.93713410e-04, 7.59772096e-02, 9.24022790e-01, 1.00000000e+00,
-6.09564253e-03, 6.22668978e-04, -2.86000000e+01, 2.96000000e+01,
1.00000000e+00, -2.86000000e+01, 1.00000000e+00, 7.60705448e-06,
9.99992393e-01, 1.00000000e+00, -6.89306742e-03, 2.95404526e-04,
3.24158843e-01, 6.75841157e-01, 1.00000000e+00, 7.03184952e-03,
2.02939054e-03, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]))
Hi, I got this problem too. For solving it, I just set the upper and lower bound of the box to (1000,-1000) instead of (0,1), and then the simulation works. You can have a try.
Hi, I got this problem too. For solving it, I just set the upper and lower bound of the box to (1000,-1000) instead of (0,1), and then the simulation works. You can have a try.
Sorry, did you direct change the box in ray lib?
finally I find how to
check the file under flow/flow/envs/merge.py or some others. There is a function named
@property def observation_space(self): """See class definition.""" return Box(low=-1000, high=1000, shape=(5 * self.num_rl, ), dtype=np.float32) set the low to -1000 and the high to 1000
finally I find how to
check the file under flow/flow/envs/merge.py or some others. There is a function named
@Property def observation_space(self): """See class definition.""" return Box(low=-1000, high=1000, shape=(5 * self.num_rl, ), dtype=np.float32) set the low to -1000 and the high to 1000
hello,I try this way,but the error above still exists, especially when I set the horizon to a high value(e.g. 1000). So I am very confused
