Sunny

Recommendation for training on new language ?

43

Any recommendations to train or fine-tune model on new language. 1. Does training for new language (E.x. arabic) will work on existing pre-trained models ? Or it has to be...

Move git under Version Control.

1

I think it would be more structured and easy approach if we move Git as a subdirectory to Version Control.

For (Psuedo) Text Reading Task

1

Hi for text reading task it instructs that: `You can use our SynthDoG 🐶 to generate synthetic images for the text reading task with proper gt_parse. See ./synthdog/README.md for details.`...

Is there any way to retain the output embeddings. Or How to get vectors from the output layer?

Hi, Is it possible to convert this weights into https://github.com/facebookresearch/llama/tree/main/llama format?

Can we integrating LongRoPE.

2

They have implemented LongRope patch for llama and mistral. Is it possible to port same into ligGPT ? https://github.com/datamllab/LongLM/tree/master