Shinichi Tanaka

I'm interested in using the recent Crello dataset with the flex-dm project. Specifically, I'm looking at revisions 4.0.0 or 5.0.0 of the Crello dataset available on Hugging Face. My question...

Why are you eliminating the influence of positional information on modalities using RoPE?

9

In your implementation, you're removing the positional information for modalities within Transfusion using the `derive_rotary_positions_from_modality_positions` function. In the Transfusion paper, information about positional encoding is only briefly mentioned in footnote...

Question: Task ordering strategy during training stage2, 3

1

Thank you for the great work on this project. I noticed that tasks are trained in a fixed order rather than being shuffled: - In stage 2, tasks follow the...