Jiacheng Liu

icon indicating a website link https://liujch1998.github.io icon indicating an email [email protected]

icon company University of Washington icon location Seattle, WA icon location PhD'ing @ UWCSE/UWNLP. Cooking @ Ai2. Private pilot. Prev: SWE @ FB/oculus, Undergrad @ UIUC, ACM ICPC World Finalist '19.

Results 3 comments of Jiacheng Liu

Support in-loop evals from oe-eval-internal request dump

Why are we missing `csqa_rc_0shot_bpb` and `socialiqa_rc_0shot_bpb` in the list?

Support in-loop evals from oe-eval-internal request dump

I got the following error when running `sciq_rc_0shot_bpb` ``` 2024-08-23T17:55:29.295911089Z 2024-08-23 10:55:29.295 jupiter-cs-aus-110.reviz.ai2.in:0 olmo.train:1005 INFO Running evaluation for 'sciq_rc_0shot_bpb'... 2024-08-23T17:55:29.368329339Z 2024-08-23 10:55:29.367 jupiter-cs-aus-110.reviz.ai2.in:0 olmo.train:1029 INFO [eval_step=1/-1] 2024-08-23T17:55:29.394690515Z 2024-08-23 10:55:29.394 jupiter-cs-aus-110.reviz.ai2.in:0...

Support in-loop evals from oe-eval-internal request dump

Thanks a lot! Other tasks ending with `_rc_0shot_bpb` ran successfully for me, only buggy one is `sciq_rc_0shot_bpb`