weave icon indicating copy to clipboard operation
weave copied to clipboard

docs(weave): Custom scorers tutorial

Open J2-D2-3PO opened this issue 9 months ago • 1 comments

Description

  • Fixes https://wandb.atlassian.net/browse/DOCS-1388

Description:
This PR adds a new tutorial to the Weave docs demonstrating how to build a custom Scorer class using the [Vale](https://vale.sh) linter. Also, this shows how to evaluate LLM outputs for style and grammar compliance using codified editorial rules (e.g. Google Developer Style Guide, write-good, etc.) which we will need to dogfood and eval our own docs editor bot ideas https://github.com/wandb/docs/blob/main/scripts/gpt-editor.py and https://github.com/wandb/weave/blob/aa169d502ab4b8e372ffe15333f84628a58002f1/docs/scripts/agentdocs.py

Testing

yarn start on local

Todo

  • [ ] Add to sidebar
  • [ ] Double check code
  • [ ] Get a good dashboard screenshot and document poking around in Weave UI

J2-D2-3PO avatar Apr 01 '25 23:04 J2-D2-3PO