prompttools
prompttools copied to clipboard
hegelai
Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
### 🚀 The feature As we add benchmarks, it would be good to cover common jailbreak scenarios. We should incorporate these benchmarks, and have auto-evals that check responses to see...
### 🚀 The feature We need a way to experiment with different chunking + ingestion strategies. For example, we have some "raw" documents we want to ingest into a vector...
### 🚀 The feature We want to integrate with Ollama: https://github.com/jmorganca/ollama Goal is to build an experiment, like the LlamaCpp example: https://github.com/hegelai/prompttools/blob/main/prompttools/experiment/experiments/llama_cpp_experiment.py ### Motivation, pitch Ollama is a new way...
### 🚀 The feature * Right now we only support adjusting the temperature parameter, it would be good to add other parameters to the left side pane * Connect to...
### 🚀 The feature Today we support huggingface inference API, which are lightweight instances for testing models. There is a different feature, called inference _endpoints_ which is for larger models...
### 🚀 The feature * A way to store inference results * A way to store old prompts along with their aggregate stats * A way to designate which version...
### 🚀 The feature * LLM-generated expected responses * Move auto-eval to a utility function * LLM chooses between multiple responses ### Motivation, pitch We want more pre-built evaluation functions...
### 🚀 The feature We need experiment classes that generalize the OpenAI experiments to any OpenAI compatible API. ### Motivation, pitch We can use this to support any completion or...
st.experimental_get_query_params was deprecated in version 1.30.0. Use [st.query_params](https://docs.streamlit.io/develop/api-reference/caching-and-state/st.query_params) instead. Streamlit Documentation [here](https://docs.streamlit.io/develop/api-reference/caching-and-state/st.experimental_get_query_params) Updated UI Design: - Hegel AI Logo (top) - Github Link (bottom)
Updated requirements.txt file to include the streamlit library and pyperclip library
