merlot_reserve
merlot_reserve copied to clipboard
rowanz
Code release for "MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound"
Hi, I installed the package following your guidance. However, when I ran the demo_video.py, it raised a ValueError: Unpack failed: incomplete input when doing **state=checkpoints.restore_checkpoint(ckpt_dir_path, target_state, step=step, prefix='ckpt_', parallel=True)** in...
The ASR transcripts in YT-Temporal 180M have a cleaned version. The cleaned transcripts have punctuation and are much more fluent than the original ASR. Does YT-Temporal 1B has such transcripts?
Hi, is there any plan to release the code for K600 classification?
Hi， Thank you for your excellent work! I have noticed that you mention the limitations of the model in your paper: “Our model only learns from 40-second long videos”. So...
Is it possible to get negative loss for each task during pretraining? Also can you share the pretraining log file (mostly the loss of each task, i.e., audio2text, audio_text_matching etc.)?
Hello dear author, Could you please release the infilled questions, i.e. the questions transformed to statements with using GPT-3? I would be especially interested in the statements for MSRVTT-QA and...
Hello, I am trying to process a dataset for training using data/process.py. Can you please share some example inputs? For example, what is the format of the `youtube_dump/{video_id}/{video_id}.v2.info.json.gz` file (in...
Hi, I have a question about the relative location for TVQA. `t_start = midpoint - segment_size * 0.5 t_end = midpoint + segment_size * 0.5 # Try to extend by...
