Naman Jain

icon indicating a website link naman-ntc.github.io icon indicating an email [email protected]

icon company UC Berkeley icon location Berkeley icon location CS Ph.D. Berkeley | AI for Code, Math, Reasoning | Previously - AWS CodeWhisperer | MSR India | CSE, IIT Bombay

Results 24 comments of Naman Jain

Added precise python backtick check

Thanks, but I do not think having python identifier is a robust solution for this. I recommend using custom extraction format to ensure reliability

Fix: memory explosion issue

Hi, thanks for the note. I would highly recommend not restricting the memory limit but reducing parallelization if you face memory issues, as it can lead to many false negatives...

Fix: memory explosion issue

> The issue is not caused by parallelization. Even with 1 process, if the generated codes contain any zombie part, the IDLE issue will show up, and LCB cannot kill...

reproduction of test time scaling

Yes, it must be Verifier