Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control

Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control copied to clipboard

I run python main.py

I got the error: Traceback (most recent call last): File "main.py", line 124, in main() File "main.py", line 76, in main predicted_state_std=predicted_state_std) File "/home/lq/Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control/control_objects/gp_mpc_controller.py", line 806, in add_memory not ('p_train' in self.dict and not self.p_train._closed): AttributeError: 'SpawnProcess' object has no attribute '_closed' Traceback (most recent call last): File "", line 1, in File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.6/multiprocessing/spawn.py", line 105, in spawn_main exitcode = _main(fd) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.6/multiprocessing/spawn.py", line 115, in _main self = reduction.pickle.load(from_parent) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.6/multiprocessing/synchronize.py", line 110, in setstate self._semlock = _multiprocessing.SemLock._rebuild(*state) FileNotFoundError: [Errno 2] No such file or directory ^CError in atexit._run_exitfuncs: Traceback (most recent call last): File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll Process Process-1: pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt Traceback (most recent call last): File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/lq/Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control/utils/utils.py", line 473, in anim_plots_2d_p if (time.time() - last_update_time) > time_between_updates:

Can you give me some suggestions?

Hi,

Can you update your python version to 3.8 (update your anaconda version if necessary), and see if you get the same error ?

Thanks.

@SimonRennotte Thanks for your quickly reply.

1. I update my python version to 3.8.
2. The project can run sucessfully a few steps, then I got the error: Traceback (most recent call last): File "main.py", line 124, in main() File "main.py", line 77, in main action, info_dict = ctrl_obj.compute_action(obs_mu=obs) File "/home/lq/Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control/control_objects/gp_mpc_controller.py", line 639, in compute_action self.check_and_close_processes() File "/home/lq/Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control/control_objects/gp_mpc_controller.py", line 985, in check_and_close_processes self.models[model_idx].initialize(params_dict_list[model_idx]) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/gpytorch/module.py", line 95, in initialize module.initialize({name: val}) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/gpytorch/module.py", line 95, in initialize module.initialize(**{name: val}) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/gpytorch/module.py", line 99, in initialize setattr(self, name, val) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1178, in setattr object.setattr(self, name, value) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/gpytorch/kernels/kernel.py", line 245, in lengthscale self._set_lengthscale(value) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/gpytorch/kernels/kernel.py", line 254, in _set_lengthscale self.initialize(raw_lengthscale=self.raw_lengthscale_constraint.inverse_transform(value)) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/gpytorch/module.py", line 103, in initialize raise RuntimeError( RuntimeError: Attempting to manually set a parameter value that is out of bounds of its current constraints, Interval(4.000E-03, 2.500E+01). Most likely, you want to do the following: likelihood = GaussianLikelihood(noise_constraint=gpytorch.constraints.GreaterThan(better_lower_bound)) Process Process-1: Traceback (most recent call last): File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/lq/Data-Efficient-Reinforcement-Learning-with-Probabilistic-Model-Predictive-Control/utils/utils.py", line 477, in anim_plots_2d_p msg = queue.get() # Read from the queue File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/queues.py", line 116, in get return _ForkingPickler.loads(res) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/site-packages/torch/multiprocessing/reductions.py", line 289, in rebuild_storage_fd fd = df.detach() File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/resource_sharer.py", line 57, in detach with _resource_sharer.get_connection(self._id) as conn: File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/resource_sharer.py", line 87, in get_connection c = Client(address, authkey=process.current_process().authkey) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/connection.py", line 502, in Client c = SocketClient(address) File "/home/lq/anaconda3/envs/py3.6-Gaussian_processes/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory

It seems you are trying to initialize the lengthscale value outside the constrainsts bound. If it is the case, you need to update the constraints on the parameter so that it makes sense. Can you let me know if it works ? If not, can you paste your config file here?

main_parameters_pendulum.json: { "env_to_control": "Pendulum-v1", "render_env": true, "verbose": true, "render_live_plots_2d": true, "save_plots_2d": false, "save_plots_model_3d": false, "save_render_env": true, "freq_iter_save_plots": 30, "number_tests_to_run": 1, "run_live_graph_parallel_process": true, "num_steps_env": 150, "random_actions_init": 20 }

params_Pendulum-v1.json: { "gp_init": { "noise_covar.noise": [1e-4, 1e-4, 1e-4], "base_kernel.lengthscale": [[0.75, 0.75, 0.75, 0.75], [0.75, 0.75, 0.75, 0.75], [0.75, 0.75, 0.75, 0.75]], "outputscale": [5e-2, 5e-2, 5e-2] }, "gp_constraints": { "min_std_noise": 1e-3, "max_std_noise": 3e-1, "min_outputscale": 1e-5, "max_outputscale": 0.95, "min_lengthscale": 4e-3, "max_lengthscale": 25.0, "min_lengthscale_time": 10, "max_lengthscale_time": 10000 }, "controller": { "target_state_norm": [1, 0.5, 0.5], "weight_state": [1, 0.1, 0.1], "weight_state_terminal": [10, 5, 5], "target_action_norm": [0.5], "weight_action": [0.05], "obs_var_norm": [1e-6, 1e-6, 1e-6], "len_horizon": 15, "exploration_factor": 3, "limit_action_change": false, "max_change_action_norm": [0.05], "num_repeat_actions": 1, "clip_lower_bound_cost_to_0": false, "include_time_gp": false }, "constraints_states": { "use_constraints": false, "state_min": [-0.1, 0.05, 0.05], "state_max": [1.1, 0.95, 0.925], "area_multiplier": 1 }, "train": { "lr_train": 7e-3, "iter_train": 15, "training_frequency": 10, "clip_grad_value": 1e-3, "print_train": false, "step_print_train": 5 }, "actions_optimizer": { "disp": null, "maxcor": 2, "ftol": 1e-15, "gtol": 1e-15, "eps": 1e-2, "maxfun": 2, "maxiter": 2, "iprint": -1, "maxls": 2, "finite_diff_rel_step": null }, "memory": { "min_error_prediction_state_for_memory": [3e-4, 3e-4, 3e-4], "min_prediction_state_std_for_memory": [3e-3, 3e-3, 3e-3] } }

Note: When I run main.py, I got the error that Pendulum-v0 is not registered. Thus, I changed Pendulum-v0 to Pendulum-v1. I only changed this name, and I did not change any other parameter.

Did you follow the installation instructions in the ReadMe?

Normally you should not have to change Pendulum-v0 to Pendulum-v1.

Can you provide the version of the packages you get when writing "pip freeze" in your virtual environment?