can-ai-code icon indicating copy to clipboard operation
can-ai-code copied to clipboard

Published 20 hours ago • verified publisher icon the-crypt-keeper

Investigate problems with ctranslate2

Open SebastianBodza opened this issue 1 year ago • 7 comments

Is it possible to investigate the problems of ctranslate2 in more detail? The library is one of the fastest and supports token streaming. Unfortunately with beam search no token streaming is possible and there the performance is quite bad :/

Is there any way to run the interview locally?

P.s. in the readme cformers2 should be ctranslate2

SebastianBodza avatar Aug 17 '23 08:08 SebastianBodza