MGM icon indicating copy to clipboard operation
MGM copied to clipboard

关于token数量的疑问

Open DimplesL opened this issue 3 months ago • 4 comments

针对llama2-7b模型,max token length为2048,按照stage2 的训练参数,一旦设置IMAGE_GRID=2和IMAGE_GLOBAL为True, image_features = torch.cat([image_feat_global, image_features], dim=1) image_aux_features = torch.cat([image_aux_feat_global, image_aux_features], dim=1) 两行最终得到的图像特征token数就变成了2880,这里token数不是超了吗?如果理解有误敬请指正。

DimplesL avatar Apr 07 '24 08:04 DimplesL