guidance icon indicating copy to clipboard operation
guidance copied to clipboard

Generated text not respecting pattern

Open Ryul0rd opened this issue 1 year ago • 0 comments

I'm trying to generate some text using a modified version of the main example and am running into an issue where the model is picking a token containing a non-numerical character for the 'chapter' variable despite that not matching the pattern. Here's my code:

prompt = '''
Tweak this proverb to apply to machine learning model instructions instead.

{{proverb}}
- {{book}} {{chapter}}:{{verse}}

UPDATED
Where there is no guidance{{gen 'rewrite' stop='- '}}
- GPT {{gen 'chapter' pattern='[0-9]' max_tokens=1}}:{{gen 'verse' pattern='[0-9]+' stop='\\n'}}
'''[1:-1]

model = guidance.llms.Transformers('stabilityai/stablelm-base-alpha-3b', device=0)
program = guidance(prompt, llm = model)
executed_program = program(
    proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
    book="Proverbs",
    chapter=11,
    verse=14,
)

print(executed_program)
print(executed_program["chapter"])

and the generated output:

Tweak this proverb to apply to machine learning model instructions instead.

Where there is no guidance, a people falls,
but in an abundance of counselors there is safety.
- Proverbs 11:14

UPDATED
Where there is no guidance, a people falls, but in an abundance of counselors there is safety.

- GPT 2::2
2:

I should also add that I saw a similar issue earlier today with pattern not working as intended so I made sure I was on the latest version just before running this just now.

Ryul0rd avatar May 18 '23 04:05 Ryul0rd