Iris Z comments

Results 38 comments of


                                            Iris Z

[DTensor] Turn on foreach implementation of optimizer for DTensor by default

@pytorchmergebot rebase

[DTensor] Turn on foreach implementation of optimizer for DTensor by default

@pytorchmergebot rebase

[DTensor] Turn on foreach implementation of optimizer for DTensor by default

@pytorchmergebot merge

[DTensor] Turn on foreach implementation of optimizer for DTensor by default

@pytorchmergebot merge

[FSDP1] Update docstring to include device_mesh arg

@pytorchmergebot rebase -s

[FSDP1] Update docstring to include device_mesh arg

@pytorchmergebot merge

[torchtitan][optim] Add fused as an option in train config

> I am curious if we have any experiments to see the performance difference with `fused=True`. Gonna talk to Tianyu to learn how to run the perf experiments on the...

[torchtitan][optim] Add fused as an option in train config

@wanchaol @awgu Added performance diff in the summary. I think we are comfortable offering this option in torchtitan?