knagrecha

icon indicating a website link kabirnagrecha.com

icon company Meta, Inc. icon location Menlo Park, CA icon location Research Scientist @ Meta (Co-Design). Prev. Netflix Research (recs + content gen) , UCSD PhD w/ Arun Kumar & Hao Zhang. Large-scale DL perf optimization.

Results 3 comments of knagrecha

Performance on GPUs and multiple GPU support

Also seeing this issue. Monitoring GPU usage shows that only one GPU is being utilized when running BERT.

StreamingDataset with DDP hangs and then crashes with NCCL timeout error

Same issue here

Hydra Release 2021.11

Also, really need to get support for PyTorch 1.10 and the corresponding torchtext. The codebase itself should be fine, even have some possible optimization opportunities (see saved_intermediates branch) but the...