multimodal topic

A curated list of Multimodal Related Research.

Eurus-Holmes

Pytorch implementation of Multimodal Neural Machine Translation(MNMT).

Eurus-Holmes

Language Models Can See: Plugging Visual Controls in Text Generation

yxuansu

An implementation of Deep Generalized Canonical Correlation Analysis (DGCCA or Deep GCCA) with pytorch.

arminarj

Neural Machine Translation with universal Visual Representation (ICLR 2020)

cooelf

Transformer-based online speech recognition system with TensorFlow 2

georgesterpu

Easily compute clip embeddings and build a clip retrieval system with them

rom1504

WIT (Wikipedia-based Image Text) Dataset is a large multimodal multilingual dataset comprising 37M+ image-text sets with 11M+ unique images across 100+ languages.

google-research-datasets

☁️ Build multimodal AI applications with cloud-native stack

jina-ai

Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

OFA-Sys