syncode icon indicating copy to clipboard operation
syncode copied to clipboard

Code generation example is broken

Open CompilerCrash opened this issue 7 months ago • 1 comments

Hi, cool project! However, the example for Python code generation from your README appears to be broken.

1) Model

WizardLM/WizardCoder-1B-V1.0 seems to be no longer available on Hugging Face, so I'd suggest replacing it.

2) Constrained generation

Running the example code with SynCode 0.4.13 gives the following error:

[2025-05-06 09:41:02,721-syncode.grammar_mask.grammar_constrainer] - --------------------------------------------------
[2025-05-06 09:41:02,721-syncode.grammar_mask.grammar_constrainer] - Parsing failed! Falling back to unconstrained decoding.
Exception: Unexpected token Token('RETURN', 'return') at line 5, column 1.
Expected one of: 
	* _INDENT

Partial code: def is_prime(n):
'''Return if prime'''

if n < 2:
return False
Parsed lexical tokens: [Token('DEF', 'def'), Token('NAME', 'is_prime'), Token('LPAR', '('), Token('NAME', 'n'), Token('RPAR', ')'), Token('COLON', ':'), Token('_NL', "\n'''Return if prime'''\n\n"), Token('LONG_STRING', '\n'), Token('_NL', "\n'''Return if prime'''\n\n"), Token('IF', 'if'), Token('NAME', 'n'), Token('LESSTHAN', '<'), Token('DEC_NUMBER', '2'), Token('COLON', ':'), Token('_NL', '\n'), Token('RETURN', 'return')]
[2025-05-06 09:41:02,721-syncode.grammar_mask.grammar_constrainer] - --------------------------------------------------

I couldn't use the original WizardCoder model, of course, but the outcome was the same for the models I tried. Also note that this error does not happen with SynCode 0.4.12.

P.S.: May I ask which version of SynCode you used for the evaluations in your paper? Thanks in advance!

CompilerCrash avatar May 06 '25 09:05 CompilerCrash

The constrained generation bug (2) has been fixed in version 0.4.14, so that part of the issue is solved.

CompilerCrash avatar May 08 '25 12:05 CompilerCrash