Metaworld
Metaworld copied to clipboard
Farama-Foundation
Baseline results on v2 benchmarks
Hi,
Since the benchmarks are moving to v2 environments, I would imagine this would change the baseline results too. Have you tried running the baselines on v2 benchmarks, and if so could you share the results? This would make it much easier to run and compare algorithms on the new version.
We definitely intend to share results for v2 as soon as we've finished running and aggregating them. @avnishn has been working on that, and I expect we will release those results within the next couple of weeks.
We definitely intend to share results for v2 as soon as we've finished running and aggregating them. @avnishn has been working on that, and I expect we will release those results within the next couple of weeks.
Great!!!
We've had baselines available in our sister repo, garage, for a while now.
I'm updating them with what I ran to get the v2 baseline results.
You can track my progress here:
https://github.com/rlworkgroup/garage/pull/2287
The updates are stable, I'm just adding documentation and tests in order to pass our code merge-ability checks.
Hello @avnishn,
Thanks for providing us your PR. Quick question: is it possible to also share the "progress.csv" files of the baseline runs? It would save a lot of time and compute resources for those who want to have the baseline curves (as we have for Figures 12~18).
Thanks!
I'll have to get back to you on that @luckeciano. Although I'd recommend re-running your results for any benchmarks that you decide to use, because there might be algorithmic differences in the implementations of these algorithms that you use vs. the ones that I wrote.
As Meta-World ownership has been transferred from the RLWorkGroup to Farama Foundation, I am closing this issue. If there are any questions or requests for features please join our Discord
