Multimodal-Transformer

Multimodal-Transformer copied to clipboard

Data pre-processing

2

Hello, I noticed in your article that the data set has been preprocessed with different methods for three modes. Could you share your code for preprocessing the original data set?

eatmorevegetables

Can you share the hyperparameter configuration? According to the settings in the paper, it cannot achieve the effect in the text. The following are the commands I set in the...

cyang810

the result is bad

7

According to the provided parameters, the model training is performed. The effect is good in the first two epochs, but over-fitting in the latter? may I know what is the...

fuziwang

想请教您咱这篇论文的多模态transformer能否应用与融合音视觉模态，其中视觉模态包括了背景图片和人脸图片特征。 如果可以，我该具体提取代码中的哪些部分进行修改。 我尝试提取了models.py，发现被hyp_params这个参数困住了，想请问能否提前给该参数都定义好，直接不使用该参数。 或者有什么好的办法，能让这个模块融入我们的代码中，十分感谢！

userLx888

Patch 1

1

I've fixed the issues related to CUDA while generating the dataloader, also I've tried replacing some code that was creating errors while transforming the data in the training procedure. Also,...

Nid989

murraykkkeed06

RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

1

hi, when I run 'python main.py', there is a bug"RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory". so i just want to know how to solve this, thanks~

diaoyudiaochan

Modal dimensions (audio, video) for the MOSI are different from values reported in paper?

1

Hi, I am noticing discrepancy in model performance between runs on latest version of MOSI from the CMU multimodal SDK and the numbers reported in the paper. Upon digging further,...

souravBhat

This model is very useful for my scenario however it appears that without using Facet I can never get the amount of AUs needed. Even with openface it is probably...

ojss

I am dealing with the imbalanced classed of the text + tabular data model. Is there a way to set up the class weights? Thank you very much!

curiousRed