Mava icon indicating copy to clipboard operation
Mava copied to clipboard

Feature/json logger

Open RuanJohn opened this issue 1 year ago • 1 comments

What?

Created a logger that can write data from the evaluator process to a structured json file.

Why?

This will enable for the usage of the EMARL repo tools library for aggregating and plotting Mava experiment data. And for the sharing of raw experiment data which is part of following the suggested evaluation protocol as outlined in the EMARL paper.

How?

Created a JSONLogger class which gets instantiated and added to the evaluator process in the Logger component of a Mava system.

Extra

Added a dictionary called environment_task_name to the make_environment functions for the debugging environments, SMAC, Flaltand, PettingZoo and RoboCup. The PettingZoo MPE examples also seem to have been removed so I re-added them in this PR as well. close #762

RuanJohn avatar Oct 04 '22 12:10 RuanJohn

Codecov Report

Merging #775 (6a38eaf) into develop (22b06e5) will decrease coverage by 0.05%. The diff coverage is 92.94%.

:exclamation: Current head 6a38eaf differs from pull request most recent head f890632. Consider uploading reports for the commit f890632 to get more accurate results

@@             Coverage Diff             @@
##           develop     #775      +/-   ##
===========================================
- Coverage    92.69%   92.63%   -0.06%     
===========================================
  Files          150      152       +2     
  Lines         8362     8449      +87     
===========================================
+ Hits          7751     7827      +76     
- Misses         611      622      +11     
Impacted Files Coverage Δ
mava/environment_loop.py 82.67% <9.09%> (-5.46%) :arrow_down:
mava/utils/loggers/base.py 88.88% <87.50%> (-0.48%) :arrow_down:
mava/utils/loggers/eval_json_logger.py 97.43% <97.43%> (ø)
mava/components/building/environments.py 88.00% <100.00%> (-0.16%) :arrow_down:
mava/components/building/loggers.py 100.00% <100.00%> (ø)
mava/components/building/system_init.py 96.62% <100.00%> (-0.04%) :arrow_down:
mava/components/updating/parameter_server.py 92.20% <100.00%> (ø)
mava/systems/ippo/config.py 100.00% <100.00%> (ø)
tests/components/building/environments_test.py 98.88% <100.00%> (ø)
tests/components/building/loggers_test.py 100.00% <100.00%> (ø)
... and 38 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov[bot] avatar Oct 11 '22 11:10 codecov[bot]