vanna icon indicating copy to clipboard operation
vanna copied to clipboard

Published 20 hours ago • verified publisher icon vanna-ai

[Feature Request] Ability to Evaluate Answers as Correct or Incorrect from Python Modules

Open ryandward opened this issue 1 year ago • 4 comments

I'm using Vanna in my Python application, and I've found that there's currently no way to evaluate the correctness of the AI's answers directly from the Python modules. This makes it difficult to provide feedback and improve the AI's performance.

ryandward avatar Jan 15 '24 03:01 ryandward

Just for clarification, you're talking about the vn.ask(..) function?

zainhoda avatar Jan 15 '24 03:01 zainhoda

Hey this feature is now available on the flask App, but might be added inside the vn.ask function as a simple y/n input for the user after user uses the ask function

ArslanS1997 avatar Jan 19 '24 09:01 ArslanS1997

Oh yeah, I was referring to the vn.ask function (Python). I have bd using it with SQLite and it's not really possible to do anything with -- it started responding with wrong answers pretty consistently.

ryandward avatar Jan 21 '24 16:01 ryandward

Oh yeah, I was referring to the vn.ask function (Python). I have bd using it with SQLite and it's not really possible to do anything with -- it started responding with wrong answers pretty consistently.

What do you have in vn.get_training_data() ?

The results will totally be dependent on what's in there

zainhoda avatar Jan 21 '24 16:01 zainhoda