GaLore icon indicating copy to clipboard operation
GaLore copied to clipboard

Published 20 hours ago • verified publisher icon jiaweizzhao

Please add Phi-2 Support

Open calebmor460 opened this issue 11 months ago • 1 comments

Attempting to use Galore to finetune a phi model yields "AttributeError: 'PhiConfig' object has no attribute 'rms_norm_eps'", which, having gotten that error on other LLM things, typically translates to "this code doesn't support Phi models"

fixing this would be incredibly nice, as It would allow people with cruddier computers to finetune LLMs

calebmor460 avatar Mar 13 '24 16:03 calebmor460