Shanmugam Ramasamy comments

Repositories
Issues
Comments

Results 26 comments of


                                            Shanmugam Ramasamy

fix bert mcore for tp, pp and sp

jenkins

fix bert mcore for tp, pp and sp

jenkins

fix bert mcore for tp, pp and sp

WandB project for tp,pp, sp both nemo legacy bert model and mcore bert model. The curves match https://wandb.ai/shanmugamr/mcore_test?workspace=user-shanmugamr

Update Jenkinsfile

jenkins

[BUG] Crash when enable --tp-comm-overlap

@erhoo82 Can you takea look into this.

Remove some duplicate code.

jenkins

Remove some duplicate code.

jenkins

Remove some duplicate code.

jenkins

Remove some duplicate code.

jenkins

[BUG] T5 extended attention mask shape mismatch with transformer engine

Hi @andrewvli , this is about the transformer engine version you are using and the type of attention you are using. There are a couple of ways to fix this...