William Zeng

Starting with Deepseek as a provider makes sense to me, as it seems to be one of the cheapest. We could consider making the provider customizable if there's a demand...

Thanks for your interest @HelixY2J ! The GPT2 config actually uses a different config class, `AsyncEvaluationConfig`, which contains some different parameter values than the other eval configs. It has the...

Hey @HelixY2J, QwQ just released hours ago, and since it performs very well as a reasoning model, I'm adding the configs ASAP to support training. My apologies for commandeering this...

It would be great if you could try running training or evaluation to confirm that they work well, and to find optimal hyperparameters for training. Reproducing their reported benchmark results...

I just merged the PR. Thanks for your contribution @jrwana !

Thanks for doing an initial analysis and bringing up the issue! I'll take a look into custom evaluation as it's definitely important to support. UPDATE: Unassigning myself for now due...

Go for it! Thanks for taking a look.

Hi @abhiramvad , could you please fix the precommit errors? You can verify it passes locally by running `pre-commit run --all-files --show-diff-on-failure`.

@abhiramvad could you fix the test error? You can run the checks locally with `pre-commit run --all-files --show-diff-on-failure`. Please re-request review when all tests pass locally.

Thank you for making this change @abhiramvad !