LLM-Finetuning-Toolkit

LLM-Finetuning-Toolkit copied to clipboard

A lot of people want LLMs to output JSON for easy parsing/post-processing A test that passes if the output is valid JSON, fails otherwise.

SinclairHudson

DRAFT FOR NOW I'm working on some documentation to help users test their LLMs. It's a really difficult task, and there's no agreed upon best-practice. This should both help guide...

SinclairHudson

example config file to run inference only on fine-tuned model

1

Is it possible to provide a config file that shows how to run inference on an already fine-tuned model? I have run the starter config, and it looks like the...

richardehughes

The command `llmtune inference [experiment_dir]` aims to provide a versatile interface for running inference on pre-trained language models, allowing users to: 1. Load and run inference on a dataset; or...

benjaminye

quickstart basic - missing qa/llm_tests:?

1

Ran: llmtune generate config llmtune run ./config.yml Things worked well (once I fixed my mistake with Mistral/huggingface repo permissions). The job ran very fast and put results into the "experiment"...

richardehughes

This improves user experience when desire output is json E.g. ```yaml prompt_stub: >- { "foo": {col_1}, "bar": {col_2} } ``` The current approach would not work as we're capturing everything...

benjaminye

- Right now debug outputs and warnings are suppressed in favor of a cleaner UI - Should leave users to choose a more verbose output by running something like ```shell...

benjaminye

How do you handle history?

2

Some of my examples of historical chat logs. How should I incorporate this into the input ideally?

angeliney

- Makes it clear that this method performs inference on the test set

benjaminye

truskovskiyk