Q&A RAG pipeline with MongoDB, Hugging Face and Llama3

[RichmondAlake]

What does this PR do?

Description of Changes: This update introduces an integrated pipeline using Retrieval-Augmented Generation (RAG) with MongoDB and Hugging Face's open-source Llama3 model for advanced question answering systems. The implementation covers the setup of a MongoDB database, data ingestion, vector search, and leveraging Llama3 for generating responses. These modifications aim to showcase how combined database and NLP technologies can improve data retrieval and processing capabilities for complex queries.

Motivation and Context: This change was motivated by the need to demonstrate practical applications of combining cutting-edge NLP models with modern database systems for educational and development purposes in the fields of AI and data science.

Dependencies:

• datasets for loading and processing the data.
• pandas for data manipulation.
• pymongo for interacting with MongoDB.
• sentence_transformers for generating text embeddings.
• transformers for accessing pre-trained models from Hugging Face.

Before submitting

• [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• [X ] Did you read the contributor guideline, Pull Request section?
• [ ] Was this discussed/approved via a Github issue? Please add a link to it if that's the case.
• [ ] Did you make sure to update the documentation with your changes?
• [ ] Did you write any new necessary tests?

Thanks for contributing 🎉!

[RichmondAlake]

@HamidShojanazeri Thanks for reviewing very quickly.

I have made the modifications and suggested changes.

Richmond

[RichmondAlake]

@jeffxtang Thanks for pointing out these issues.

I have addressed them and added screenshots where appropriate.

[jeffxtang]

Thanks @RichmondAlake for the changes and the screenshots. I tried following steps under Atlas UI (there're 10!) but got lost - I already logged in and completed the survey then I clicked Try Free and it asked me to Sign up/in again - with Sign in I saw the Overview - Create a deployment page, but the instructions with 10 steps page was no longer visible... The RAG and Llama 3 parts look good, but I'm afraid many users will probably be stuck in the Mongodb setup...

Is there another simpler way to to showing MongoDB for this use case? A while ago I downloaded some Mongodb locally on Mac to use it with Llama 2, which worked pretty easily (the setup process and integration).

I can approve this PR if you're fine with the current DB setup process.

[RichmondAlake]

@jeffxtang thanks for your comment.

Please move forward with merging the PR.

I will work on the experience with the docs with our internal team.

Thanks for the review