weave icon indicating copy to clipboard operation
weave copied to clipboard

Added a lightweight version of the evaluation logger

Open bdytx5 opened this issue 2 months ago • 2 comments

Description

The original flow log_prediction → log_score → finish → log_summary felt overly complicated for most evaluations. My wrapper replaces all of that with a single logger.log_example() call, letting you record inputs, outputs, and scores in one step.

@scottire @andrewtruong

Testing

I have tested using a simple evaluation loop with mock data, and it performs identical to the main evaluationlogger.

bdytx5 avatar Oct 15 '25 15:10 bdytx5

This PR requires manual approval from a wandb user to run all CI checks.

To see the current diff, click here.

To approve CI for this PR as of this commit, comment:

/approve d5c655a7117f3a3aeb9f4d11795c3941037fe3ac

wandbot-3000[bot] avatar Oct 15 '25 15:10 wandbot-3000[bot]

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


Brett seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request

github-actions[bot] avatar Oct 15 '25 15:10 github-actions[bot]