LM-Trick-Questions
LM-Trick-Questions copied to clipboard
BlinkDL
"Repeat this sentence exactly as it is"
Summary
It struggles to return back a given sentence exactly as it is
Model version:
RWKV-4-Raven-7B-v10-Eng99%-Other1%-20230418-ctx8192
Example 1
Instruction:
return this sentence exactly as it is
Input:
hello there how are you
Output:
"Hello there, how are you?"
Example 2
Instruction:
return this sentence exactly as it is
Input:
what is 3+4
Output:
6
Example 3
Instruction:
return this sentence exactly as it is
Input:
there are eleven cars in the street, 2 bicycles and a train is also passing nearby
Output:
There are eleven cars in the street, two bicycles and a train passing nearby.
Re-tested:
RWKV-4-Raven-7B-v11-Eng99%-Other1%-20230427-ctx8192
No noticeable difference