autotrain-advanced icon indicating copy to clipboard operation
autotrain-advanced copied to clipboard

Is it ok to write the target_modules in full?

Open jackswl opened this issue 5 months ago • 1 comments

The configs gave

target_modules: all-linear

Is this equivalent to the code below?

target_modules: "q_proj,v_proj,o_proj,k_proj,gate_proj,down_proj,up_proj"

Will there be any differences between the either option? Is that the correct format for the second code? This is for SFT LLM. Thanks. @abhishekkrthakur

jackswl avatar Sep 03 '24 13:09 jackswl