guidance
guidance copied to clipboard
Generated text not respecting pattern
I'm trying to generate some text using a modified version of the main example and am running into an issue where the model is picking a token containing a non-numerical character for the 'chapter' variable despite that not matching the pattern. Here's my code:
prompt = '''
Tweak this proverb to apply to machine learning model instructions instead.
{{proverb}}
- {{book}} {{chapter}}:{{verse}}
UPDATED
Where there is no guidance{{gen 'rewrite' stop='- '}}
- GPT {{gen 'chapter' pattern='[0-9]' max_tokens=1}}:{{gen 'verse' pattern='[0-9]+' stop='\\n'}}
'''[1:-1]
model = guidance.llms.Transformers('stabilityai/stablelm-base-alpha-3b', device=0)
program = guidance(prompt, llm = model)
executed_program = program(
proverb="Where there is no guidance, a people falls,\nbut in an abundance of counselors there is safety.",
book="Proverbs",
chapter=11,
verse=14,
)
print(executed_program)
print(executed_program["chapter"])
and the generated output:
Tweak this proverb to apply to machine learning model instructions instead.
Where there is no guidance, a people falls,
but in an abundance of counselors there is safety.
- Proverbs 11:14
UPDATED
Where there is no guidance, a people falls, but in an abundance of counselors there is safety.
- GPT 2::2
2:
I should also add that I saw a similar issue earlier today with pattern not working as intended so I made sure I was on the latest version just before running this just now.