ciaranby

[Feature Request] Metrics that require knowledge of input.

1

For generate_until output type tasks, I would like to be able to register metrics that are computed using the input prompt to the LLM as well as as the reference...

Add ability to inject OpenAI client to LM

1

Add abillity to inject OpenAI client into the OpenaiCompletionsLM class. This can be useful since the [official python client](https://github.com/openai/openai-python) library has multiple client implementations such as the AzureClient which can...

[CLI] Error - Cannot specify both target_size and target_fraction

5

### Describe the bug When running: wandb artifact cache cleanup 1GB as per the documentation we get the following error: ValueError: Cannot specify both target_size and target_fraction Seems to be...

Fix partial caching of openai models

6

* Caching key is generated using all kwargs otherwise there is a cache miss. * also a couple of linting changes made by the code formatter