generative-ai icon indicating copy to clipboard operation
generative-ai copied to clipboard

[Feat]: Add sample of Vertex AI Generative AI Evaluation service with PydanticAI

Open ghchinoy opened this issue 11 months ago • 0 comments

Is your feature request related to a problem? Please describe.

I'd like to see a PydanticAI agent evaluated with the Vertex AI Generative AI Evaluation service SDK so that I can utilize VAI Gen AI Evals on my agents.

Describe the solution you'd like

A python example using the Vertex AI Generative AI Evaluation service SDK and a PydanticAI agent showing single tool use, tool trajectory use, and final response evaluations, similar to the existing notebooks for LangGraph and CrewAI.

Describe alternatives you've considered

A notebook, though colab's google-auth and PydanticAI's google-auth versions clash (bug filed elsewhere).

Additional context

No response

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

ghchinoy avatar Jan 26 '25 22:01 ghchinoy