oneflow icon indicating copy to clipboard operation
oneflow copied to clipboard

Add a switch for memory share strategy

Open Yipeng1994 opened this issue 3 years ago • 2 comments

Add a switch for the memory share strategy. It would be off by default. It solves this issue https://github.com/Oneflow-Inc/oneflow/issues/9508

Yipeng1994 avatar Dec 02 '22 15:12 Yipeng1994

Speed stats:
GPU Name: GeForce GTX 1080 









❌ OneFlow resnet50 time: 139.9ms (= 13990.2ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 161.7ms (= 16173.9ms / 100, input_shape=[16, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.16 (= 161.7ms / 139.9ms)

OneFlow resnet50 time: 84.7ms (= 8473.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 101.0ms (= 10104.4ms / 100, input_shape=[8, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.19 (= 101.0ms / 84.7ms)

OneFlow resnet50 time: 57.6ms (= 11514.9ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 77.8ms (= 15565.4ms / 200, input_shape=[4, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.35 (= 77.8ms / 57.6ms)

OneFlow resnet50 time: 43.8ms (= 8768.1ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 71.1ms (= 14225.6ms / 200, input_shape=[2, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.62 (= 71.1ms / 43.8ms)

OneFlow resnet50 time: 39.9ms (= 7982.9ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
PyTorch resnet50 time: 68.1ms (= 13624.1ms / 200, input_shape=[1, 3, 224, 224], ddp, world size=2)
✔️ Relative speed: 1.71 (= 68.1ms / 39.9ms)

github-actions[bot] avatar Dec 02 '22 16:12 github-actions[bot]

View latest API docs preview at: https://staging.oneflow.info/docs/Oneflow-Inc/oneflow/pr/9509/

github-actions[bot] avatar Dec 02 '22 16:12 github-actions[bot]