Steven Krawczyk (Hegel AI)
Steven Krawczyk (Hegel AI)
**Describe the bug** Because the Transport struct requires urls, you cannot pass the uri of a local docker container, which is a very common use case. **To Reproduce** Create an...
### 🐛 Describe the bug Running into the following on colab notebook: `AttributeError: 'AccelerateSFTTrainer' object has no attribute 'make_experience'`  ### Which trlX...
# Overview of Change OpenAI has a new API format that uses a client object, so we need to change the way we monkey patch logging ``` from openai import...
### 🚀 The feature Request from potential user: "There are two main aspects, 1) adjusting prompts that changing semantic words does not trigger hallucination, 2) the prompt itself is such...
### 🚀 The feature We need an experiment or example for LlamaIndex. This will be similar to the work we need to do for LangChain and MindsDB. ### Motivation, pitch...
### 🚀 The feature We need a SQL validator for evaluation, like what we have for JSON and Python. We can use this package: https://pypi.org/project/sqlvalidator/ ### Motivation, pitch Many folks...
### 🚀 The feature We need to add benchmark test sets so folks can run on models / embeddings / systems A few essentials: * BEIR for information retrieval *...
### 🚀 The feature Today, we only support link sharing for 1 instruction and 1 prompt, and we don't capture configuration like temperature and other variables. First, we should support...
### 🚀 The feature As we add benchmarks, it would be good to cover common jailbreak scenarios. We should incorporate these benchmarks, and have auto-evals that check responses to see...