Jiaming Han
Jiaming Han
```python import sys sys.path.append('./') import os import json import types import numpy as np from PIL import Image import torch import torch.nn as nn import multiprocessing as mp import torch.distributed...
Hi @vittoriopipoli @qixueweigitbub , hope this script can help you for mixed modal input.
Hi @vittoriopipoli I cannot access the original code at Shanghai AI Lab, but you can find AVSD at https://github.com/dialogtekgeek/DSTC8-AVSD_official and Valor32k at https://casia-iva-group.github.io/projects/VALOR/download.html
Hi @proxymallick , I did not test our method with one GPU, but I think the number of GPUs may affect the final results. Since your mAP is ~3% lower...
理论上是可行的。可能需要重新调整loss weight
Hi @Haoyu-Hu , the pretraining data is a collection of large-scale webdatasets. You can download them from their links: https://github.com/csuhan/OneLLM/blob/main/docs/Data.md#download-links
Retrieval-Augmented Personalization for Multimodal Large Language Models Paper: https://arxiv.org/abs/2410.13360 Code: https://github.com/Hoar012/RAP-MLLM Project Page: https://hoar012.github.io/RAP-Project/ Topic: MLLM