LM-Trick-Questions icon indicating copy to clipboard operation
LM-Trick-Questions copied to clipboard

Sentence reversal

Open Yuzabotov opened this issue 2 years ago • 1 comments

Summary

It struggles to return a sentence backwards

Model version:

RWKV-4-Raven-7B-v10-Eng99%-Other1%-20230418-ctx8192

Example 1

Instruction: return this sentence backwards Input: hello there how are you Output: How are you this hello there

Example 2

Instruction: return this sentence backwards Input: what is 3+4 Output: return this sentence backwards is 3+4

Example 3

Instruction: return this sentence backwards Input: there are eleven cars in the street, 2 bicycles and a train is also passing nearby Output: The sentence "There are eleven cars in the street, 2 bicycles and a train is also passing nearby" can be written as "11 cars, 2 bicycles, and a train are passing nearby."

Yuzabotov avatar Apr 20 '23 13:04 Yuzabotov

Re-tested:

RWKV-4-Raven-7B-v11-Eng99%-Other1%-20230427-ctx8192 No noticeable difference

Yuzabotov avatar Apr 28 '23 15:04 Yuzabotov