Add Cookbook Notebook for Function Calling and Text Embeddings Demonstration

[andycandy]

Overview

This PR introduces a new notebook that serves as a cookbook demonstration for integrating text embeddings with dynamic function calling in an email organization system. The notebook showcases how to leverage modern language model techniques to classify and manage emails effectively. By combining semantic text embeddings and automated function execution, this example provides a robust template for building intelligent email assistants.

Key Features

• Text Embedding for Semantic Analysis:
  The notebook utilizes a get_embedding function to convert email content into high-dimensional vector representations. These embeddings capture the semantic meaning of the text, enabling the system to differentiate between categories such as "important" and "not_important" based on content similarity.

• Centroid-Based Classification:
  Using the initialize_category_centroids and classify_email_using_centroids functions, the notebook computes and compares normalized centroids of predefined email categories. This approach allows for accurate classification of incoming emails by measuring cosine similarity between the email's embedding and the category centroids.

• Dynamic Function Calling:
  The system demonstrates dynamic function invocation by automatically triggering specific actions (e.g., add_as_important or add_as_not_important) based on the classification outcome. This function calling mechanism is central to creating a responsive and adaptive email workflow.

• Interactive Learning and Adaptability:
  The notebook includes a learn_email function that allows users to provide corrections or updates to the email classification. The system continuously refines its classification accuracy by learning from new examples and recalculating the centroids.

• User Interaction via Gemini Chat:
  An interactive chat session is set up, enabling users to input commands that directly modify email classifications. The session supports commands like "student portal mails are not important", facilitating on-the-fly teaching and corrections. This interactive component illustrates the practical application of function calling and embedding in a real-world scenario.

Why This Notebook is a Great Addition

• Practical Demonstration:
  The notebook provides a complete, end-to-end example of how to build an intelligent email organizer. It is an ideal reference for developers implementing similar functionality in their projects.

• Modern Techniques:
  By incorporating text embeddings and dynamic function calls, this example leverages cutting-edge methods in natural language processing and automation, making it a valuable resource for learning and experimentation.

• Adaptability:
  The interactive nature of the notebook allows for continuous learning and refinement, showcasing how systems can evolve with user feedback. This adaptability is crucial for developing robust AI-driven applications.

• Extensibility:
  The clear structure and modular design of the functions (e.g., embedding generation, centroid initialization, email classification, and action triggering) make it easy for developers to extend and integrate these methods into larger systems.

We believe that adding this notebook to our examples collection will not only enrich our repository but also serve as an excellent teaching tool for developers interested in state-of-the-art AI techniques for email management.

Next Steps

• Merge this PR to add the notebook to the cookbook collection.
• Encourage team members to review and experiment with the example to further enhance its capabilities.

Thank you for considering this addition!

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[andycandy]

@markmcd @Giom-V Could you please review this?

[Giom-V]

@andycandy I'm not ignoring this PR, but considering how many other PR I reviewed today I'm keeping this one for tomorrow to be able to review it more rested.

[andycandy]

All good @Giom-V please take your time

[Giom-V]

@andycandy I have to say I love you example, especially since it deals with a use case I would really benefit from (I have other automations doing similar things) and it uses embeddings which I am not extremely familiar with.

That said I have to say I was a bit disappointed when I realized I would have to do the gmail integration by myself as I was already starting to think about new cool use-cases (preparing email drafts for all my important emails for ex.). I think it's Ok that way as it's first and foremost an embeddings and FC example, but if you already wrote that part and are willing to, I think it would be great to add it into the notebook as well (maybe with a way to run it in "read-only mode" in case you just want to test and don't want to take risks). If you can't, maybe explain that you won't show that part in the notebook intro, to manage expectations.

In any case, thanks a lot, that's an amazing contribution that I'm sure will help a lot of developers. I made a few comments but all in all it's great!

[andycandy]

@Giom-V, Could you please give me a day for this? I might need to look for colab-mail integration docs to start working on it and I'll surely get that done. Thank you so much for this review; it meant a lot!

[Giom-V]

@Giom-V, Could you please give me a day for this? I might need to look for colab-mail integration docs to start working on it and I'll surely get that done. Thank you so much for this review; it meant a lot!

Take all the time you need :)

[andycandy]

@Giom-V All right! After some testing and digging, I realized that following this demo would require users to fetch their Gmail API keys from GCP. I’m wondering if that might be a bit much for some users, since not everyone might be comfortable doing that. Do you think I should continue down this path, or would it make sense to consider an alternative?

[Giom-V]

@Giom-V All right! After some testing and digging, I realized that following this demo would require users to fetch their Gmail API keys from GCP. I’m wondering if that might be a bit much for some users, since not everyone might be comfortable doing that. Do you think I should continue down this path, or would it make sense to consider an alternative?

I think it's fine. It's a complex demo after all and we've already done it in other examples such as the virtual try-on one.

Since it will be quite some work we can also add the notebook as it is for now, and let you work on a follow-up to add the details about the gmail integration.

[andycandy]

@Giom-V All right! After some testing and digging, I realized that following this demo would require users to fetch their Gmail API keys from GCP. I’m wondering if that might be a bit much for some users, since not everyone might be comfortable doing that. Do you think I should continue down this path, or would it make sense to consider an alternative?

I think it's fine. It's a complex demo after all and we've already done it in other examples such as the virtual try-on one.

Since it will be quite some work we can also add the notebook as it is for now, and let you work on a follow-up to add the details about the gmail integration.

Alright then, I'll start working on it!

[andycandy]

@Giom-V, here is the prototype I made, please look through this once: https://colab.research.google.com/drive/1D4Cy9mW2Jm-Ggmu-s9eZLgJDAjnpkLFL?usp=sharing