Clay Fortune

@FrankLeeeee Thank you, that's great! I'm looking forward to the RLHF pipeline.

Hi, @taishiciR All the models are handled by the strategy, so I utilized the load function from the strategy class. https://github.com/hpcaitech/ColossalAI/blob/ea0b52c12ee7598fe126ed0d8b0557f7e8a0e999/applications/ChatGPT/examples/train_prompts.py#L75 However, as you mentioned, I think the functionality is...

Hi @araffin, Thank you for your response! I've attached the complete code, as requested, and I hope it has been helpful. The missing imports have been added, and [Ray](https://docs.ray.io/en/latest/ray-overview/installation.html) is...

@araffin Thank you for your feedback! I'm glad the sample code worked well for you. Thank you for fixing the bug! > There seems to be a big overhead compared...

Thank you for your comment! I have noticed that in `def step_wait`, `ray.get` is around 10 times slower (mp: O(1.0e-5) sec. < ray: O(1.0e-4) sec.) in comparison to utilizing `recv...

@george-adams1 No, it hasn't been integrated. It seems that the overhead caused by using Ray is not acceptable for some use cases when compared to multiprocessing. If you take a...

+1