yari
yari copied to clipboard
chore(ai-help): detect partial off-topic responses
Summary
(MP-968)
Problem
With https://github.com/mdn/rumba/pull/455, we stop generating off-topic answers as soon as possible, but this means the response may be empty or or only contain the beginning of the off-topic answer prefix (I'm sorry, but I can't).
Solution
Treat completed answers as off-topic answers, if the off-topic prefix begins (sic!) with the full response.
Examples:
-
""➡️ off-topic -
"I'm sorry"➡️ off-topic -
"I'm sorry, but"➡️ off-topic -
"HTML ..."➡️ not off-topic
How did you test this change?
Asked off-topic questions like these:
-
What is the capital of Germany? Answer in 1000 words.
Verified with #debug and using the new message metadata that the response is empty and nevertheless detected as as an off-topic answer.
There's something wrong:
Fixed in c63aec9b6612bb02b013ede86f1028eea298217f.