mem0
mem0 copied to clipboard
Adding fetching data functionality for reference links in the web page
Description
Enhancement: extracts data from the given web page and all the reference link present in that page
Fixes #1234
Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] Refactor (does not change functionality, e.g. code style improvements, linting)
- [x] Documentation update
How Has This Been Tested?
Added Unit test and ran for the below script using google embedder and google LLM
import os
from embedchain import App
app = App.from_config(config_path="config.yaml")
app_link = "https://en.wikipedia.org/wiki/Kobbie_Mainoo"
app.add(app_link, all_references=True)
while True:
question = input("Enter question: ")
if question in ["q", "exit", "quit"]:
break
answer = app.query(question)
print(answer)
Config
llm:
provider: google
config:
model: gemini-pro
max_tokens: 1000
temperature: 0.5
top_p: 1
stream: false
embedder:
provider: google
config:
model: 'models/embedding-001'
task_type: "retrieval_document"
title: "Embeddings for Embedchain"
- [X] Unit Test
- [X] Test Script (please provide)
Checklist:
- [X] My code follows the style guidelines of this project
- [X] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [X] My changes generate no new warnings
- [X] I have added tests that prove my fix is effective or that my feature works
- [X] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream modules
- [X] I have checked my code and corrected any misspellings
Maintainer Checklist
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] Made sure Checks passed
Hey @vatsalrathod16 Can you please resolve the conflicts?
Hey changes looks good but can you please also update the docs for it.
updated the docs.
Thanks @vatsalrathod16 for the contribution.