Ishaan Datta
Ishaan Datta
@lopagela addressed the changes, please check once
> I've had this problem, too. Is there a solution? Was getting this error- got resolved by removing cpu offloading... hoping for an explanation. Also, any suggestions to increase token...
@shaowei-su I'm using the bf16 version you linked. @lhl thank you for sharing this! I'm currently using tp=4 pp=6 as we're aiming for context lengths > 64k. Just to clarify,...