Steps About Generated Code Solutions Post-processing

[MrBlack0220]

Hi，thanks for the amazing work! I wank to ask about the detailed steps about generated code solutions post-processing when testing one solution. (e.g. After a code solution was generated, did you truncate it by stop tokens?(e.g. : “\nclass”, “\ndef”, “\n#”)) Thanks for your reply!

[xksteven]

We didn't truncate the solutions from the model outputs. We did spot check a few of the solutions to observe if we could find any common stopping patterns or criterion and couldn't at that time.

Let me know if you have further questions.