Questions about Ray implementation of PPO

[cryoco]

Discussed in https://github.com/hpcaitech/ColossalAI/discussions/3606

^{Originally posted by cryoco April 19, 2023} I've seen 2 ray implementations of PPO in this repo, #3195 and #3309 . The former makes the experience maker and trainer decoupled, whereas the latter's models are still shared by experience maker and trainer. I'd like to know which implementation will be maintained in the future. Thank you!