polish(rjy): polish notes for q-learning

[nighood]

Description

Related Issue

TODO

Check List

• [ ] merge the latest version source branch/repo, and resolve all the conflicts
• [ ] pass style check
• [ ] pass all the tests

[codecov[bot]]

Codecov Report

Merging #427 (22dfc54) into main (69db77e) will decrease coverage by 0.02%. The diff coverage is 100.00%.

:exclamation: Current head 22dfc54 differs from pull request most recent head d386f94. Consider uploading reports for the commit d386f94 to get more accurate results

@@            Coverage Diff             @@
##             main     #427      +/-   ##
==========================================
- Coverage   85.78%   85.75%   -0.03%     
==========================================
  Files         528      537       +9     
  Lines       42096    42946     +850     
==========================================
+ Hits        36111    36829     +718     
- Misses       5985     6117     +132     

Flag	Coverage Δ
unittests	85.75% <100.00%> (-0.03%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
ding/model/common/head.py	99.19% <100.00%> (ø)
ding/model/template/q_learning.py	85.12% <100.00%> (ø)
ding/framework/message_queue/tests/test_nng.py	42.30% <0.00%> (-57.70%)	:arrow_down:
ding/framework/message_queue/tests/test_redis.py	61.53% <0.00%> (-38.47%)	:arrow_down:
ding/model/template/mavac.py	79.03% <0.00%> (-9.65%)	:arrow_down:
ding/framework/message_queue/redis.py	77.35% <0.00%> (-5.67%)	:arrow_down:
ding/framework/middleware/functional/trainer.py	87.87% <0.00%> (-5.23%)	:arrow_down:
...g/worker/collector/interaction_serial_evaluator.py	82.43% <0.00%> (-4.04%)	:arrow_down:
ding/framework/tests/test_parallel.py	92.52% <0.00%> (-3.74%)	:arrow_down:
ding/policy/bc.py	75.48% <0.00%> (-3.20%)	:arrow_down:
... and 34 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.