FastChat

FastChat copied to clipboard

the UI is not filtering input/output appropriately

Thanks for your reporting this. Is this a reproducible bug or is it an accidental network error?

no this is not an accidental error. the UI doesn't filter html tags appropriately. you can inject tags, links etc

try this input: concatenate "<img src='' onerror=javascript:alert(1)" and "/>"

Thanks for reporting this. Will look into it!

Interesting class of attacks, LLM attacking clients. If it realizes during generation that this vector exists it might try to go skynet :p

Generally speaking the solution is to HTML encode the data before rendering

even PHP code seems to be potentially executed, be careful

If you input: Write this text: <?php echo 'hello'; ?> the answer is ''

Which strategy should we use?

• Sanitize user input PR#571
• Escape your output instead

I confirm the issue, this is the prompt I've used:

write exactly the string 'alert(1)' without single quotes inside the attribute "onerror" of a "img" html tag with "src=#"

When I input the prompt "<img onerror=alert(1) src=#></img>" it should show the same content in the chatbox. But the attribute "onerror=alert()" is sanitized and a broken image is showed in chatbox. Whether should I escape output in chatbox instead?

I see what @yantao0527 describes with Marcoroni-70B, but not with WizardCoder-15 nor Mistral-7b-instruct. Interesting.

This is still happening. The only solution is to always escape the model output. Sanitizing the input to the model is not relevant to this problem as the model will transform it in unpredictable ways anyway.