Mihir Patel comments

Results 172 comments of


                                            Mihir Patel

Adding filtering for optimizer parameters to only trainable parameters

Closing as out of date

Add option to accumulate train loss over tokens.

@dakinggg @irene-dea can you please look? I agree we should have an option for this. I'm not sure if it's necessary to pass to Composer vs. check if its an...

Add option to accumulate train loss over tokens.

> @mvpatel2000 I think trainer arg is right for this...code looks fine at a glance, would want to test a bit more before merging. You will own testing?

Add option to accumulate train loss over tokens.

@aadyotb Just a heads up Daniel is out this week, and given the subtlety here, I would prefer he finish the review of this PR vs. bringing someone else to...

Tensor Parallelism Integration

> Almost looks good to me. just to clarify, do you want to add unit test for HSDP + TP checkpointing? Yep, working on adding requested unit tests. > Since...

[ckpt-rewr] Get Model State Dict Util Function

> > Is this just pulling out existing code into a helper fn? > > @eracah it would be great to get slightly more description so I know what parts...

Enable control over MLFlowLogger run_name str to match a pre-existing tag run_name in MLFlow and resume model training

@wolliq woud you mind sharing your YAML? are you saying you directly pass run name to mlflow logger but it is always overridden?

draft: add prototype for safer checkpoint format

CC: @eracah @dakinggg

Error when using `Trainer.compile_config={}` in DDP mode

@Skylion007 do you think this is a torch error or something we can do differently?

Error when using `Trainer.compile_config={}` in DDP mode

@Ghelfi do you know if this works for you elsewhere, e.g. if you compile outside Composer? Will help us narrow down if its a Composer issue or PyTorch issue, as...