Wizyoung comments

Repositories
Issues
Comments

Results 22 comments of


                                            Wizyoung

[BUG] deepspeed overlap_comm data race

This issue is not fixed yet.

报错： RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

Just add a cuda stream for device separation may fix this. ```python stream = torch.cuda.Stream() # you can place it into __init__ with torch.cuda.stream(stream): output = self.model.generate(xxx) stream.synchronize() ``` This...