Arabizi/ Franco-Arabic text translated to English

[omnipervius]

What is your question?

I am trying to translate Arabizi to English. This is the romanized version of the Arab language which is often used on social-media. Is this model able to translate from arabizi or do you know other model already trained for that purpose? I want to avoid normalization of the text before passing it to the model, but if anybody can propose some good normalization model (arabizi to arabic) it would also be helpful. :)

What have you tried?

We are classifiyng the input as Arabic or Moroccan language, but the model is not able to transalte a single word as it execpt the classic arabic symbols.

[hwang136]

Hey, my friend how do you know which special token represents the Arabic language? there are 21 special tokens ending with Arab and I do not know which one I should use.