omi
omi copied to clipboard
BasedHardware
Friend app should be able to chat with and have context from your gmail ($1000)
Is your feature request related to a problem? Please describe. Users want to be able to chat with their emails. Gmail doesn't have native AI features and is not smart and doesn't do it well. We need to let people to connect to Gmail through a plugin and upload their emails into Friend app and that data should be used same way we currently use memories.
Describe the solution you'd like You need to make a "Connect Gmail" plugin for Friend app, in which the user will be able to authorize their gmail app and export all the data and then import it into Friend app.
Mails should be imported properly and vectorized in the backend, when chatting the user should be able to ask questions about their email.
A new section of the app with your Gmails indexed, should be in there.
Question
- A simple Gmail Chat API search works? instead of indexing all this stuff? or why not?
Learn how to create plugins here
Additional context This is a paid task. Reward is $1000 in cash. Simply link your PR with this task and we will check it asap. Check how Contributions and bounties work here If no one is assigned to this task, then no one is working on it. We don't lock bounty tasks unless you've created PRs before.
Hello @kodjima33, I am working on it. I have experience with it and flutter. can you assign for me? Thank you
This is 90% not a flutter task.
@ologunB @heidy00 let me know if you are still interested.
Hello @josancamon19 , please contact with me via telegram. https://t.me/vsheidy This is my Telegram.
let's chat on discord https://discord.gg/9puRjM2g @josancamon19
I can't add your discord. vsheidy000 it is my username
@Washyl @samarsheikh001 @ologunB @heidy00 haven't seen any code so far - can someone please do a PR, we will check and if it's good, will assign the issue?
For now, the issue is still unassigned due to lack of code here
Question: A simple Gmail Chat API search works? instead of indexing all this stuff? or why not?
Answer: Simple Gmail Chat API search should works, cause Gmail is so generous on rate-limiting[1]. ver1 of this plugin should be use Gmail API to retrieve relevant emails ( messages in term of Gmail API ) whenever users ask, we don't need to index all of emails.
[1]: https://developers.google.com/gmail/api/reference/quota
@beastoin yeah i think for initial implementation we can just use chat API
But eventually we definitely will need to index emails in app's database to give more context for ai
@beastoin yeah i think for initial implementation we can just use chat API
But eventually we definitely will need to index emails in app's database to give more context for ai
keep in mind that some users do have thousands of mails in their inbox.
i'd suggest doing the mail handling serverside at the plugin server and searching mails with a non-ai based approach, fetching them regularily using IMAP. maybe it could be useful to fetch only the last 100 mails or so.
thus, the google quota won't be a problem, and there won't be many resources required.
@kodjima33 @josancamon19 completed the plugin to chat with mail
On the implementation side of it, 🟢 supports gmail auth similar to notion-crm and webhooks to handle triggers 🟢 to support the chat with mail, found it best to have it trigger on chat messages instead of memory, so added chat-query as a capability for external integrations. 🟢 The webhook is hooked up with langchain's mail toolkit for querying for mails. It also supports sending mails which can also be added if required.
wanted to get feedback and info on how to submit and have this plugin added since it requires the webhook to be hosted aswell.. will be cleaning and making a pr shortly
