[FEATURE-REQUEST] Integrate Neo4j Graph Database as a Knowledge Source for TinyPersons via LangChain

[felipe-nunes]

Enable TinyPersons to first consult a Neo4j graph database through a LangChain-based chain before interacting with the LLM.

This will provide:

• Access to accurate and updated information stored in the graph database, reducing potential hallucinations from LLM-generated answers.
• A dual approach where structured responses come from the graph, and fallback answers are provided by the LLM for information not present in the graph.

The idea here is to

• Create a connector that allows TinyTroupe to interface with Neo4j, utilizing LangChain to include database querying in its chain workflow.
• Develop a custom chain where TinyPersons access the graph database, process results, and use the LLM only when necessary.
• Integrate GraphRAG (Retrieval-Augmented Generation with Graphs) to enable TinyPersons to produce accurate answers that incorporate structured data from Neo4j.

Benefits:

• Improved accuracy and reliability of responses by leveraging real-time data from a graph database.
• Enhanced knowledge sharing among TinyPersons, ensuring consistent and factual responses
• Flexibility to use LLMs for queries beyond the scope of the graph, maintaining comprehensive query handling.
• Leverage Advanced Graph Algorithms
• Scalability that ensures consistent performance as TinyTroupe grows in its usage and data requirements
• Utilizing Neo4j as a knowledge graph can provide a structured way to represent relationships between entities, enabling more contextually rich responses. TinyPersons will have a better understanding of the interconnected data, leading to more precise answers and insights.

[paulosalem]

Thanks for starting this. We'll prepare a base class from which this and other connectors can then be created for grounding.

[OriginalGoku]

Would be great to do this without having to depend on other packages such as LangChain. This is certainly a very important feature

[paulosalem]

@felipe-nunes , in this recent v0.4.0 release, we have introduced grounding data connectors, you can see them here: https://github.com/microsoft/TinyTroupe/blob/main/tinytroupe/agent/grounding.py

For now, this includes:

• BaseSemanticGroundingConnector: a local vector-based store.
• LocalFilesGroundingConnector: provides access to local files in a given folder.
• WebPagesGroundingConnector: provides access to Web pages.

If you want, you can try extending GroundingConnector in order to have Neo4j as a grounding provider along similar lines.

Then, to actually make this available to agents, you'd need to implement an additional TinyMentalFaculty, similar to FilesAndWebGroundingFaculty -- for example, a Neo4jGroundingFaculty. Or maybe a more general grounding faculty, say a GeneralGroundingFaculty. Mental faculties are defined here: https://github.com/microsoft/TinyTroupe/blob/main/tinytroupe/agent/mental_faculty.py

Faculties are used during agent instantiation as a parameter, so that the agent has the said mental capability available.