matters-server icon indicating copy to clipboard operation
matters-server copied to clipboard

Unable to translate long article

Open tx0c opened this issue 2 years ago • 1 comments

for example, 四萬字長文 seems always failed to translate

TODO:

  1. need to figure out what's the translation limit
  2. fix the limit, make sure all articles be translatable; (the longest article in the platform currently is 200K (20萬字))

tx0c avatar Aug 19 '22 07:08 tx0c

The Cloud Translation API is optimized for translating smaller requests. The recommended maximum length for each request is 5K characters (code points). However, the more characters that you include, the higher the response latency. For Cloud Translation - Advanced, the maximum number of code points for a single request is 30K. Cloud Translation - Basic has a maximum request size of 100K bytes. An alternative option is to use batch translations to send more content per request as a long running operation.

https://cloud.google.com/translate/quotas#content

We might need to use batchTranslateText API, but it's more complex to call since needs input files.

robertu7 avatar Aug 29 '22 05:08 robertu7