Fedor Laputin

Results 2 comments of Fedor Laputin

have you solved this problem?

> found out this is caused by `module_wrap_policy` function in the `FSDP(trainer.model)`. ![image](https://private-user-images.githubusercontent.com/43416079/295629824-65f2ceda-7204-4944-8fff-32d9e4a66f3e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzE5MjMxNjksIm5iZiI6MTczMTkyMjg2OSwicGF0aCI6Ii80MzQxNjA3OS8yOTU2Mjk4MjQtNjVmMmNlZGEtNzIwNC00OTQ0LThmZmYtMzJkOWU0YTY2ZjNlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDExMTglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMTE4VDA5NDEwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWU0YTMxMTI2MGNjYzJjMjY4NTk0MmFjOGIwMTNlODk4Nzk3NThmODBiNjg1ZjlmMGE0YzhjYjhlZTRiMDc4YzcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.NQeQ4FWl8j3vFe2yOR2mntCU8ZrPCXOKZCGL96XsmkI) > > Peft wrapped model passed a `None` as the module class variable. ![image](https://private-user-images.githubusercontent.com/43416079/295630864-d98ca9a9-68de-4470-b757-e2cce281af76.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzE5MjMxNjksIm5iZiI6MTczMTkyMjg2OSwicGF0aCI6Ii80MzQxNjA3OS8yOTU2MzA4NjQtZDk4Y2E5YTktNjhkZS00NDcwLWI3NTctZTJjY2UyODFhZjc2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDExMTglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMTE4VDA5NDEwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTBhMWMyMzlkNzNlZjNiNDkyMmQzNzg4ZjBmYjdiMDNjZGQ0YWQ2ZDk3YTVlZDc2Mzk0ZWQ4MmRjMTlmMzc2ZmEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.bGiHzsJkf2Zi0b38r2NMWBGwfx5oJRXyiYzxZHTKOnc) > > After...