Bruno Maga

icon indicating a website link https://brunomaga.github.io/

icon company @microsoftresearch icon location Switzerland icon location Large-scale AI @ Microsoft Research

Results 1 issues of Bruno Maga

[BUG] High memory usage on first GPU, despite perfectly-balanced stages in pipeline

2 comment

**Describe the bug** When using pipelining (with or without `LayerSpec` inside `PipelineModule`), the first GPU seems to have a considerably higher memory consumption, compared to the other ones. This is...

bug
training