Finetune of scripts/train/yamls/finetune/mpt-7b_dolly_sft.yaml with alibi and triton emits sketchy log

[tginart]

Tried setting max_seq_len to 5k and added alibi to attn config with Triton flash attn.

Otherwise using: scripts/train/yamls/finetune/mpt-7b_dolly_sft.yaml.

Finetuning seems to be working but it does emit this sketchy log during pretraining. Wondering if anyone has seen this?

Map:  12%|█▏        | 1750/15015 [00:01<00:08, 1573.51 examples/s]Token indices sequence length is longer than the specified maximum sequence length for this model (5201 > 5120). Running this sequence through the model will result in indexing errors