bigcodebench icon indicating copy to clipboard operation
bigcodebench copied to clipboard
verified publisher icon bigcode-project

Add status code to mock response for BigCodeBench tasks 211 and 215

Open hvaara opened this issue 1 year ago • 1 comments

This PR adds 200 OK status code for the mocks in task 211 and 215.

TODO

  • [ ] Verify model passes task with new behavior
  • [ ] Check other tasks for same issue

Fixes #33

hvaara avatar Sep 14 '24 22:09 hvaara

@hvaara Let me know how it goes with the model verification :) If it works, please put it into tools/fix_0.2.1 and point to the data to v0.1.3.

terryyz avatar Oct 06 '24 08:10 terryyz

@terryyz I'm working on testing resolution with starcoder2-15b where the bug was detected. Which fix file, BIGCODEBENCH_VERSION and BIGCODEBENCH_NEW_VERSION should I be targeting?

hvaara avatar Feb 19 '25 00:02 hvaara

@terryyz This is ready to be merged assuming I'm making changes to the correct fix file.

hvaara avatar Feb 19 '25 22:02 hvaara

Thanks @hvaara! Could you move the part to a file called fix_v023.py, which will be a patch based on the v0.2.3 release.

terryyz avatar Feb 20 '25 16:02 terryyz

@terryyz Thanks for the review! I moved the changes to a new file and updated the patch versions of BIGCODEBENCH_VERSION and BIGCODEBENCH_NEW_VERSION.

hvaara avatar Feb 20 '25 17:02 hvaara