yari icon indicating copy to clipboard operation
yari copied to clipboard

chore(ai-help): detect partial off-topic responses

Open caugner opened this issue 1 year ago • 1 comments

Summary

(MP-968)

Problem

With https://github.com/mdn/rumba/pull/455, we stop generating off-topic answers as soon as possible, but this means the response may be empty or or only contain the beginning of the off-topic answer prefix (I'm sorry, but I can't).

Solution

Treat completed answers as off-topic answers, if the off-topic prefix begins (sic!) with the full response.

Examples:

  • "" ➡️ off-topic
  • "I'm sorry" ➡️ off-topic
  • "I'm sorry, but" ➡️ off-topic
  • "HTML ..." ➡️ not off-topic

How did you test this change?

Asked off-topic questions like these:

  • What is the capital of Germany? Answer in 1000 words.

Verified with #debug and using the new message metadata that the response is empty and nevertheless detected as as an off-topic answer.

caugner avatar Mar 26 '24 18:03 caugner

There's something wrong:

Fixed in c63aec9b6612bb02b013ede86f1028eea298217f.

caugner avatar Mar 27 '24 12:03 caugner