langkit
langkit copied to clipboard
whylabs
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance me...
The example on this page appears to want version 0.0.31 to use local models. [https://github.com/whylabs/langkit/blob/main/langkit/examples/Local_Models.ipynb](https://github.com/whylabs/langkit/blob/main/langkit/examples/Local_Models.ipynb) However, it looks like the code to make this happen is only in the main...
A lot of things have changed between the main branch and the workflow branch. It reimplements all of the metrics from the main branch with a different Workflow, Metric based...
How can I use my own domain specific prompt for response hallucination detection function call in langkit?
After [this pr](https://github.com/whylabs/langkit/pull/233), models are lazily initialized when the first prediction occurs. That leads to the cached artifacts being retrieved for every prediction, which adds latency. Also, this means that...
With the changes on [this PR](https://github.com/whylabs/langkit/pull/233), local models are supported for the default model (martin-ha's toxic comment model), but are not supported for any of the detoxify's models. It would...
This enables passing different versions of `MoritzLaurer` zero-shot classification: MODEL_SMALL = "MoritzLaurer/xtremedistil-l6-h256-zeroshot-v1.1-all-33" MODEL_BASE = "MoritzLaurer/deberta-v3-base-zeroshot-v1.1-all-33" MODEL_LARGE = "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33" See added tests on how to use it.
Had an extremely difficult time trying install the Langkit module with a venv built with python version 3.12.1 on Windows 10 64-bit machine. There was an error with building the...
If you want to import a metric or the metrics module like: ```Python from langkit import toxicity, from langkit import llm_metrics ``` By default, it downloads the models from the...
You can pre-download the vader lexicon but we need to also support not making external http calls. Lets skip this line conditionally on env var: https://github.com/whylabs/langkit/blob/f4f2404d4d58b4c46ce003f054af700d730e534f/langkit/sentiment.py#L39 Alternatively, consider replacing this...
Without changing LangKit how do you successfully override the name of the prompt column. We need a working example for this.