markprompt-js icon indicating copy to clipboard operation
markprompt-js copied to clipboard

AI response may contain broken char - ��

Open johnd0e opened this issue 1 year ago • 0 comments

Describe the bug

When streaming response from AI that includes unicode symbols (e.g. cyrillic), sometimes I observe �� instead of some random character.

To Reproduce

Steps to reproduce the behavior:

  1. Go to https://markprompt.com/<your_login>/starter/playground
  2. Enter prompt like 'Output some text. Translate your response to Russian'
  3. Run this (or another) prompt several times (usually 2-3 is enough)
  4. Most probably some of responses contain at least one broken character (random).

Additional context

I think that may occur when some chunk during streaming breaks inside codepoint, thus instead of 1 valid codepoint we get 2 invalid. Though, I never experience that in other ChatGPT-related services.

johnd0e avatar Sep 07 '23 20:09 johnd0e