babyagi icon indicating copy to clipboard operation
babyagi copied to clipboard

Enable metadata tracking with Aim

Open roubkar opened this issue 1 year ago • 2 comments

Hey, Ruben from Aim here! 👋

First of all thanks for this awesome project!

After experimenting with babyagi I found that integrating Aim can be extremely helpful for prompts exploration and comparison.

How can Aim enhance babyagi observability?

Prompts comparison

Here is an example view where it is possible to explore and compare the outputs for the same objective by different models (gpt-3.5-turbo and text-davinci-003):

Screen Shot 2023-04-13 at 22 24 02

Generated tokens count comparison

The same example for generated tokens count:

Screen Shot 2023-04-13 at 22 27 46

Log messages for debugging

Screen Shot 2023-04-13 at 22 32 26

View the entire session terminal logs

Screen Shot 2023-04-13 at 22 33 56

roubkar avatar Apr 13 '23 18:04 roubkar

Hey @yoheinakajima and @francip, any thoughts on this? Would like to hear your feedback on this 🙏

roubkar avatar Apr 23 '23 13:04 roubkar

would like to see this as well

letsgoduke avatar May 11 '23 19:05 letsgoduke