OsebeSammi comments

Repositories
Issues
Comments

Results 1 comments of


                                            OsebeSammi

[BUG] Tensor are not on the same device when enable cpu activation offload

Faced similar issue when offloading to CPU using config "zero_optimization": { "stage": 3, "offload_param": { "device": "cpu", "buffer_count": 5, "buffer_size": 1e8, "max_in_cpu": 1e9 }, "offload_optimizer": { "device": "cpu" } }...