Ning Wang comments

Repositories
Issues
Comments

Results 4 comments of


                                            Ning Wang

RuntimeError: Detected at least one parameter gradient is not the expected DDP bucket view with gradient_as_bucket_view=True.

do you have a minimal repro?

Tensor Parallelism Integration

Almost looks good to me. just to clarify, do you want to add unit test for HSDP + TP checkpointing? If so, do you want to add that in this...

[Not for merge] refactor the object_store.download_object retry and add more retry

> Thanks for the PR! Only one minor comment in the PR. However, if `get_file` calls `download_object_or_file`, should the rest of the codebase use `get_file` instead of `download_object_or_file`? @b-chu ,...

[Not for merge] refactor the object_store.download_object retry and add more retry

i will not merge this PR for now, because i just find it regresses the auto-resume. With auto-resume, it will try to `get_file(checkpoint_path)` no matter if checkpoint_path exits or not...