griptape icon indicating copy to clipboard operation
griptape copied to clipboard

Add Graph Store Drivers

Open Kornspan opened this issue 1 year ago • 2 comments

Describe your changes

Additional Context: This implementation allows future integrations with other graph databases.

Changes Made:

  • BaseGraphStoreDriver: Abstract class for graph store operations.
  • FalkorDBGraphStoreDriver: Implements the base class for FalkorDB.
  • Unit Tests: Comprehensive tests for FalkorDBGraphStoreDriver.
  • Documentation: Updated with details and usage examples.
  • Changelog: Added entries for the new features.
  • Dependencies: Added FalkorDB to pyproject.toml.

Issue ticket number and link

Closes #915


📚 Documentation preview 📚: https://griptape--1006.org.readthedocs.build//1006/

Kornspan avatar Jul 22 '24 12:07 Kornspan

@Kornspan from the read the docs build:

Installing dependencies from lock file

pyproject.toml changed significantly since poetry.lock was last generated. Run `poetry lock [--no-update]` to fix the lock file.

Seems to be a source of issues on the unit tests as well.

collindutter avatar Jul 22 '24 21:07 collindutter

Codecov Report

Attention: Patch coverage is 72.28916% with 46 lines in your changes missing coverage. Please review.

Files Patch % Lines
...ptape/drivers/graph/falkordb_graph_store_driver.py 77.50% 20 Missing and 7 partials :warning:
griptape/drivers/graph/base_graph_store_driver.py 58.69% 18 Missing and 1 partial :warning:

:loudspeaker: Thoughts on this report? Let us know!

codecov[bot] avatar Jul 23 '24 14:07 codecov[bot]

Hi can you comment what error occurred in the mkdocs? @collindutter @vachillo

galshubeli avatar Sep 09 '24 13:09 galshubeli

@galshubeli here's the output:

INFO    -  Cleaning site directory
INFO    -  Building documentation to directory: /home/docs/checkouts/readthedocs.org/user_builds/griptape/checkouts/1006/_readthedocs/html
INFO    -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
  - griptape-cloud/index.md
ERROR   -  Error reading page 'reference/griptape/artifacts/index.md': No module named 'griffe.collections'
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/__main__.py", line 366, in <module>
    cli()
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/__main__.py", line 284, in build_command
    build.build(cfg, dirty=not clean)
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/commands/build.py", line 310, in build
    _populate_page(file.page, config, files, dirty)
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/commands/build.py", line 167, in _populate_page
    page.render(config, files)
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocs/structure/pages.py", line 285, in render
    self.content = md.convert(self.markdown)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/core.py", line 357, in convert
    root = self.parser.parseDocument(self.lines).getroot()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/blockparser.py", line 117, in parseDocument
    self.parseChunk(self.root, '\n'.join(lines))
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/blockparser.py", line 136, in parseChunk
    self.parseBlocks(parent, text.split('\n\n'))
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/markdown/blockparser.py", line 158, in parseBlocks
    if processor.run(parent, blocks) is not False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings/extension.py", line 125, in run
    html, handler, data = self._process_block(identifier, block, heading_level)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings/extension.py", line 196, in _process_block
    handler = self._handlers.get_handler(handler_name, handler_config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings/handlers/base.py", line 458, in get_handler
    module = importlib.import_module(f"mkdocstrings_handlers.{name}")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.11.9/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings_handlers/python/__init__.py", line 3, in <module>
    from mkdocstrings_handlers.python.handler import get_handler
  File "/home/docs/checkouts/readthedocs.org/user_builds/griptape/envs/1006/lib/python3.11/site-packages/mkdocstrings_handlers/python/handler.py", line 14, in <module>
    from griffe.collections import LinesCollection, ModulesCollection
ModuleNotFoundError: No module named 'griffe.collections'

Note that you can run the build process yourself with make docs.

collindutter avatar Sep 10 '24 17:09 collindutter

@collindutter I encountered this error while running it on the dev and main branch as well. Do you know what the possible cause might be?

galshubeli avatar Sep 11 '24 09:09 galshubeli

@galshubeli Are your dependencies up to date? Can you try:

make install/all
make docs

I do get quite a few DeprecationWarnings but you can ignore those.

Separately, I want to apologize for the lack of attention our team has given to this PR. We're definitely interested in incorporating Graph Store Drivers into Griptape and have already started some initial work ourselves. However, we haven't yet fully figured out how they fit into the bigger picture. They will likely play a key role in RagEngine, but we need more time to work through the details.

Given this uncertainty, it's hard for us to provide meaningful feedback on the PR right now. For the time being, it might make sense to close the PR and revisit it once we've solidified our approach. When we're ready, we'd love to get your input on it!

CC @Kornspan

collindutter avatar Sep 11 '24 22:09 collindutter

Hey all, I'm going to close this PR for the reasons mentioned above. Once we're ready to support Graph Drivers we'll reach back out!

collindutter avatar Oct 01 '24 16:10 collindutter