llm-foundry icon indicating copy to clipboard operation
llm-foundry copied to clipboard

Published 20 hours ago • verified publisher icon mosaicml

Script to automatically split off eval set

Open mattyding opened this issue 1 year ago • 0 comments

What changes are proposed in this pull request?

See go/sweeps-eval for more context. Corresponding MAPI changes are contained in https://github.com/databricks-mosaic/mcloud/pull/4562.

How is this tested?

Added unit tests. Screenshot 2024-09-16 at 1 05 50 AM

The predecessor to the unit tests was this bash script I used to manually inspect the file outputs. Tests are the same as unit tests, but it invokes the scripts/... file instead of the command_utils/... one. Confirmed file outputs looked OK. _test.txt

Everything passes: Screenshot 2024-09-16 at 12 56 37 AM

mattyding avatar Sep 16 '24 08:09 mattyding