BasedHardware
Summaries like in granola
granola ai has amazing summaries. Why is it amazing:
- [ ] summaries are fluid (not a fixed template)
we need a summary that is actualy useful
you need to
- Propose a prompt and explain why is it great (ideally just figure out what prompt granola is using)
- implement the new prompt
- propose a way of how to combine manual + automated notes in omi ai app
Bounty: $500
@kodjima33, please send me the link to the meeting transcript with Granola and Omi for analysis.
I plan to reverse engineer it and improve our prompts accordingly—ideally, I want to figure out what prompt Granola is using.
To be honest, it will be very hard to compete with Granola, which has a $20M+ round and focuses exclusively on this task. I believe there’s not just one prompt but rather a series of machine learning layers trained for specific tasks.
What we can do is follow best practices and improve our prompts.
It would be great if you could share good references to get started.
Hi, I sent mu thought to Nik about that already. So will duplicate it here for everyone:
Honestly, this isn’t something we can casually whip up over a weekend. Sure, in a weekend we might come up with some clever prompt for GPT, but that’ll be a temporary hack we’ll end up throwing away anyway.
@beastoin @kodjima33
Regarding making something like Granola:
If you want to achieve a truly solid result, we’ll need to fine-tune our own model using LoRA—a smart technique that simplifies training significantly.
However, we’ll definitely need some decent hardware: • GPU: at least an RTX 3080 (ideally a 4090) • CPU: around 16 cores • RAM: about 32 GB minimum
Additionally, we’ll require a dataset of roughly 500–1000 good-quality text-to-summary pairs.
I’m actively working on fine-tuning my personal project models right now, and it’s genuinely exciting stuff!
If you’re serious about this project and ready to allocate the necessary time and budget, I’d be happy to build this out for you.
Just keep in mind: someone needs to gather or provide the dataset—I’m not particularly eager to do all that myself. In a worst-case scenario, we could generate some synthetic data to fill gaps if necessary.
But if you just need a quick prompt-based solution, I’d prefer to leave that task to a prompt engineer and focus on more complex problems we need to solve.
I’d also suggest attaching at least a few examples of high-quality recaps from Granola as references — otherwise, it’s really hard to guess what “amazing summaries” means.
My only concern is that they’ve raised a $20M round and have a team of over 10 people working on this — not sure how we can match that with a single prompt, to be honest.
As mentioned earlier, I’m happy to help with model training.
not the best experience, for most of my conversations it auto-selects "insight extractor" and doesn't show anything
@beastoin how does it selects the app to summarize? Only based on "prompt" or it would also analyze the outcome, then see if it makes sense, and if yes, then selects?
I believe we need to do 2 checks instead of one
@kodjima33 , also manual selection would be great. I would like to be able to select manually. That way, the app I want would appear first.
otherwise if it chooses randomly every time or even if it chooses the most logical one, the most logical application may change, so it will be a different experience every time.
I think the simpler the better from the user's perspective.
For example, there may be 2 options;
- auto select (who is the most make sense)
- or manual select
@kodjima33 I only download it when we publish a release on github :) testflight is for iphone?
Or did I misunderstand? By manual I mean that after selecting it once, the application we selected will always appear.
@beastoin yeah, manual choice should be saved forever - currently it's not saved forever
changes needed:
1/ better app selection prompt on auto / first-time selection - this should be based on the outcome(the result of the summarization) as well https://github.com/BasedHardware/omi/blob/main/backend/utils/llm.py#L1583-L1646
2/ keep the last manually selected app and use it for the next summary.
3/ bug: the conversation's title gets overridden the second time it’s summarized https://github.com/BasedHardware/omi/pull/2178/files#diff-f1358c888913f0733414834ed5fe83203a5e07bbd9f8ab690a42c9c6ad1b3f49L51
@mdmohsin7 could you help with these changes ^
discussion needed:
4/ users are asking for multiple app summarization @smian1 , please tell us more detail about your inside ;)
--
also guys, what do you think about the granola app on omi? i'm mimicking the prompt, trying to match it 1-1 with the granola ai app, hoping it’ll be more useful for summarization.
