Steven Krawczyk (Hegel AI)

### 🚀 The feature We need a way to experiment with different chunking + ingestion strategies. For example, we have some "raw" documents we want to ingest into a vector...

Ollama Integration

3

### 🚀 The feature We want to integrate with Ollama: https://github.com/jmorganca/ollama Goal is to build an experiment, like the LlamaCpp example: https://github.com/hegelai/prompttools/blob/main/prompttools/experiment/experiments/llama_cpp_experiment.py ### Motivation, pitch Ollama is a new way...

UI Features

1

### 🚀 The feature * Right now we only support adjusting the temperature parameter, it would be good to add other parameters to the left side pane * Connect to...

### 🚀 The feature Today we support huggingface inference API, which are lightweight instances for testing models. There is a different feature, called inference _endpoints_ which is for larger models...

### 🚀 The feature * A way to store inference results * A way to store old prompts along with their aggregate stats * A way to designate which version...

### 🚀 The feature * LLM-generated expected responses * Move auto-eval to a utility function * LLM chooses between multiple responses ### Motivation, pitch We want more pre-built evaluation functions...

### 🚀 The feature We need experiment classes that generalize the OpenAI experiments to any OpenAI compatible API. ### Motivation, pitch We can use this to support any completion or...