magmadnn icon indicating copy to clipboard operation
magmadnn copied to clipboard

[FEATURE REQUEST] Easier integration of model parallelism

Open Dando18 opened this issue 5 years ago • 0 comments

Is your feature request related to a problem? Please describe. Models are not distributed in parallel by default (i.e. when they exceed device memory). Thus the user must first encounter an Out of Memory error and then employ some model parallelism techniques.

Describe the solution you'd like The Model class or Compute Graph should be able to detect this and distribute accordingly.

Dando18 avatar Sep 13 '19 15:09 Dando18