uptrain

uptrain copied to clipboard

Support for LLMs hosted on AWS Bedrock

2

Hey Can we include AWS bedrock LLM support as well. Claude models on Bedrock cannot be used otherwise.

bithikaj

Support for LLMs running on Vertex AI

1

Can you please add support for Google Cloud Vertex AI platform? Thanks!

verdverm

Take up any use case where Uptrain can be used to evaluate the pipeline and create a descriptive Python notebook with code for the same. You can create a folder...

Dominastorm

MT-Bench (Multi-Turn Benchmark) is the benchmark to evaluate multi-turn dialogues, typically between a user and an LLM.

Dominastorm

Allow users to use the Mamba LLM architecture to generate responses and perform evaluations

Dominastorm

Creating a New Project with a bad key results in null values

1

**Describe the bug** After creating a New Project with a bad key for OpenAI GPT-3.5-Turbo, the results are returned as Null. I was expecting to get a bad key response...

cfrenchi

New Project missing .jsonl file will not indicate it as a required file on Safari

1

**Describe the bug** Followed the steps for the Dashboard outlined in [Getting Started](https://docs.uptrain.ai/dashboard/getting_started) and successfully built the project and hosted it locally through docker. (Great work, the script went smooth!)...

cfrenchi

MMLU (Massive Multitask Language Understanding) is a benchmark designed to measure knowledge acquired during pretraining by evaluating models exclusively in zero-shot and few-shot settings.

Dominastorm

Add tests for checking if all the managed evals are working fine or not

sourabhagr

There is a lot of repetition in the operator classes in 'uptrain/operators'. Create a common super class.

Dominastorm