Li Bo
Yes, it's pretty correct! I suggest you use DC mode and use Video pretrained weights. You could see via our web demo, the backend model is Video-LLaMA7B-DC. Remember to put...
1. inference could be hosted at a 24GB GPU, actually 16GB is enough for bf16 mode. 2. it could be, but we didnt tested it. You could also try the...
3. For image embedding, you could hack with this line: https://github.com/Luodian/Otter/blob/9b34a4467581869c67dae7ea2b970f8e6b201d3c/otter/modeling_otter.py#L732 As for text embedding, it's in the `forward` function inside the `modeling_llama.py` or `modeling_mpt.py`, you could see `modeling_llama.py` at...
hi the MPT1B-init is only for init to train Otter-MPT-1B model. It’s not being trained and directly migrated from OpenFlamingo-1B, but with added special tokens. So evaluating this weight may...
@pufanyi Could Fanyi take a look at this issue, I thought we did the OTTER-MPT7B evaluation and report good numbers on COCO.
so basically you could `pip install otter_ai`.
I was considering relaxing the data format and let it supports loading images from local path so you can incrementally add to a pool and let dataloader sample from the...
ok I see it's a bug from our side. we overlooked some config files dont have this attribute. Could you start with `luodian/OTTER-MPT1B-RPJama-Init`?
This one should be fine since it's our latest used model.