Error when running MT1 SAC baseline with garage

[NagisaZj]

Hi @avnishn. Thank you for your efforts in running and providing the Meta-World baselines! I meet a problem while running your baseline codes, and need some help.

I am trying to run the baseline codes you provided in #352. I installed the Garage repo at that commit, and tried to run mtsac+mt1 by running the following command (I copied the metaworld_examples folder to my working dir):

python ./metaworld_examples/mtsac_metaworld_mt1.py

Then it gives the following error:

2021-12-28 10:43:16 | [mtsac_metaworld_mt1_seed=1_env_name=pick-place-v2] Obtaining samples...

Sampling [####################################] 100%
Traceback (most recent call last): File "./metaworld_examples/mtsac_metaworld_mt1.py", line 122, in mtsac_metaworld_mt1() File "/home/zj/.local/lib/python3.6/site-packages/click/core.py", line 1128, in call return self.main(*args, **kwargs) File "/home/zj/.local/lib/python3.6/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/home/zj/.local/lib/python3.6/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/zj/.local/lib/python3.6/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/data2/zj/garage/src/garage/experiment/experiment.py", line 369, in call result = self.function(ctxt, **kwargs) File "./metaworld_examples/mtsac_metaworld_mt1.py", line 118, in mtsac_metaworld_mt1 trainer.train(n_epochs=epochs, batch_size=batch_size) File "/data2/zj/garage/src/garage/trainer.py", line 396, in train average_return = self._algo.train(self) File "/data2/zj/garage/src/garage/torch/algos/mtsac.py", line 290, in train policy_loss, qf1_loss, qf2_loss = self.train_once() File "/data2/zj/garage/src/garage/torch/algos/sac.py", line 241, in train_once policy_loss, qf1_loss, qf2_loss = self.optimize_policy(samples) File "/data2/zj/garage/src/garage/torch/algos/sac.py", line 423, in optimize_policy qf1_loss, qf2_loss = self._critic_objective(samples_data) File "/data2/zj/garage/src/garage/torch/algos/sac.py", line 367, in _critic_objective alpha = self._get_log_alpha(samples_data).exp() File "/data2/zj/garage/src/garage/torch/algos/mtsac.py", line 193, in _get_log_alpha 'The number of tasks in the environment does ' ValueError: The number of tasks in the environment does not match self._num_tasks. Are you sure that you passed The correct number of tasks?

It seems that running mtsac_metaworld_mt10.py and mtsac_metaworld_mt50.py is okay and does not give any errors. It would be great if you can provide some help!

[avnishn]

It seems that people have had a lot of interest in running mt1 examples.

We didn't publish the results for these, as we didn't have the time to run results for our rerelease. I'll talk with the other garage maintainers, and see about updating the example on that branch, however for now, please assume that these MT1 examples are not supported.

Thanks, Avnish

[NagisaZj]

Got it! Thank you for your timely reply.

I am running mt1 as I am trying to reproduce the single task results on v2 environments in the Meta-World paper (Figure 11 in https://arxiv.org/pdf/1910.10897.pdf ). By the way, are goals randomly sampled for each episode or kept fixed throughout training in Figure 11's results?

[avnishn]

Oh those are not the files you want to use then to get those results.

What you're looking for instead is ppo_metaworld.py and sac_metaworld.py.

On that same branch.

[reginald-mclean]

As Meta-World ownership has been transferred from the RLWorkGroup to Farama Foundation, I am closing this issue. If there are any questions or requests for features please join our Discord