Translation example with ctranslate2's Translator.

[uahmed93]

trafficstars

As we have added support for HF model translation via CrossFit, we are working towards performance improvement with ctranslate2. This work depends on adding support for ctranslate2 in CrossFit, and then will need to create pipeline for this work in NDC.(Draft PR)

With a workaround for ctranslate2 in CrossFit, huge performance improvement was seen. On single GPU, following is the performance :

Experiment	Standalone pytorch inference	Standalone + ctranslate2	Crossfit+ctranslate2
Inference time	~1hr 50mins	23min 54sec	6min 29sec (including extra processing for workarund : 3sec)
BLEU score	-	0.9585	0.9586

BLEU score was calculated w.r.t Standalone pytorch inference on 74058 sentences.