[zero] trivial zero optimizer refactoring

[yhna940]

📌 Checklist before creating the PR

• [ ] I have created an issue for this PR for traceability
• [ ] The title follows the standard format: [doc/gemini/tensor/...]: A concise description
• [ ] I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here. if you have any plots/diagrams/screenshots/tables, please attach them here.

It's not that important, but it seems more appropriate to call a method to modify the contents than to directly refer to the private member variable (_averaged_gradients) in self._grad_store. And very minor refactoring.

💥 Checklist before requesting a review

• [ ] I have linked my PR to an issue (instruction)
• [ ] My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
• [ ] I have performed a self-review of my code
• [ ] I have added thorough tests.
• [x] I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

• [x] 🌝 Yes, I do.
• [ ] 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

[github-actions[bot]]

The code coverage for the changed files is 92%.

Click me to view the complete report

Name                                                          Stmts   Miss  Cover
---------------------------------------------------------------------------------
colossalai/zero/sharded_optim/bookkeeping/gradient_store.py      24      3    88%
colossalai/zero/sharded_optim/low_level_optim.py                311     25    92%
---------------------------------------------------------------------------------
TOTAL                                                           335     28    92%

[github-actions[bot]]

Your pre-commit check failed, follow the steps to run pre-commit on your file for code style consistency.

1. install pre-commit via "pip install pre-commit"
2. install pre-commit hooks via "pre-commit install"
3. run pre-commit on file with format error via "pre-commit run --files path" by replacing "path" with the actual file path
4. commit and push to your branch

View your job at https://github.com/hpcaitech/ColossalAI/actions/runs/4240123889. Read our "CONTRIBUTING.md" for more reference to the code style.

[yhna940]

sort all imports (python)................................................Failed
- hook id: isort
- files were modified by this hook

Fixing /fsx/home-yhna/test/ColossalAI/colossalai/zero/sharded_optim/sharded_optim_v2.py

yapf formatter...........................................................Passed
clang formatter......................................(no files to check)Skipped
check yaml...........................................(no files to check)Skipped
check for merge conflicts................................................Passed
check for case conflicts.................................................Passed
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing colossalai/zero/sharded_optim/sharded_optim_v2.py

fix end of files.........................................................Passed
mixed line ending........................................................Passed

It seems like a per-commit error occurs where I haven't modified it. can you check it?

[binmakeswell]

sort all imports (python)................................................Failed
- hook id: isort
- files were modified by this hook

Fixing /fsx/home-yhna/test/ColossalAI/colossalai/zero/sharded_optim/sharded_optim_v2.py

yapf formatter...........................................................Passed
clang formatter......................................(no files to check)Skipped
check yaml...........................................(no files to check)Skipped
check for merge conflicts................................................Passed
check for case conflicts.................................................Passed
trim trailing whitespace.................................................Failed
- hook id: trailing-whitespace
- exit code: 1
- files were modified by this hook

Fixing colossalai/zero/sharded_optim/sharded_optim_v2.py

fix end of files.........................................................Passed
mixed line ending........................................................Passed

It seems like a per-commit error occurs where I haven't modified it. can you check it?

Hi, @yhna940 This is normal and has been fixed automatically by precommit, you can just sync the code. Thanks.

[github-actions[bot]]

The code coverage for the changed files is 92%.

Click me to view the complete report

Name                                                          Stmts   Miss  Cover
---------------------------------------------------------------------------------
colossalai/zero/sharded_optim/bookkeeping/gradient_store.py      24      3    88%
colossalai/zero/sharded_optim/low_level_optim.py                311     25    92%
---------------------------------------------------------------------------------
TOTAL                                                           335     28    92%

[binmakeswell]

@yhna940 Thank you very much for your contribution!