Zach Mueller comments

Results 411 comments of


                                            Zach Mueller

trafficstars

[FSDP] support activation offloading with FSDP

CC @pacman100

Dataloader yields wrong sequence when resuming training

Can you try updating your accelerate version to see if we fixed it in the prior releases?

Feature request: ability to truncate datasets on split rather than pad

Nice idea! Would you like to attempt a PR with your solution? :)

feat: Add data class for fsdp config and use it along with argument parser

I suppose we can do so. i think it *might* be better to have users use the raw accelerate FSDP plugin here, let me think on this!

feat: Add data class for fsdp config and use it along with argument parser

A dataclass similar to the `AcceleratorConfig` would be the way to go here. @kmehant would you like to take a stab at it? :)

feat: Add data class for fsdp config and use it along with argument parser

I suppose there actually wouldn't be a harm in just importing the FSDP one. My only fear would be how easy it is to pull up the docs/get to the...

Update stateful_callbacks state before saving checkpoint

@pedrobrs can you rebase from main? This should fix the failing tests

Update stateful_callbacks state before saving checkpoint

looks like this logic broke some of the tests in test_trainer.py: ``` tests/trainer/test_trainer_callback.py::TrainerCallbackTest::test_stateful_duplicate_callbacks ``` Looks like this is because this change doesn't actually wind up restoring the state?

Update stateful_callbacks state before saving checkpoint

cc @ArthurZucker for final 🚀

Integrate Liger (Linkedin GPU Efficient Runtime) Kernel to Trainer

@JasonZhu1313 if you run `make fixup` it should fix the quality tests :) Otherwise as Marc said, let us know when we're okay to land this and we'll merge it...