fiddler-auditor

fiddler-auditor copied to clipboard

Need to test example notebooks as part of nightlies

iterix

Facing issue while installing using pip on windows

1

I am trying to install fiddler-auditor on my windows system having python version 3.8.10. I tried below command to install the fiddler-auditor. pip install fiddler-auditor==0.0.2 I created a fresh virtual...

himanshu034

Support for HF models

1

Currently auditor supports LLM providers via Langchain. However, we need to support HuggingFace models as well so that users can audit open-source or fine-tuned models as well

iterix

Add ChainEval Class

1

Right now there is only one evaluation class which is LLMEval. This class can only evaluate Langchain's LLM objects. I suppose we need to add support for langchain's Chain objects...

mstfatay

External API calls, particularly to open-ai services are being made in unit-tests. Removing this dependency will 1. Allow contributors to run unit-tests with out provinf API keys 2. Reduce cost...

iterix2