ml-ferret icon indicating copy to clipboard operation
ml-ferret copied to clipboard

Published 20 hours ago • verified publisher icon apple

Upgrading from FlashAttention (1.x) to FlashAttention-2, the fucntion…

Open abdur75648 opened this issue 1 year ago • 0 comments

Upgrading from FlashAttention (1.x) to FlashAttention-2, flash_attn_unpadded_qkvpacked_func has been renamed and gives error. Reference - https://github.com/Dao-AILab/flash-attention/issues/318#issuecomment-1756153770

Also, during finetuning we're getting runtime RuntimeError: TensorBoardCallback in FERRETTrainer requires tensorboard to be installed. Either update your PyTorch version or install tensorboardX. So, tensorboard is an important dependency

abdur75648 avatar Jan 24 '24 19:01 abdur75648