mage icon indicating copy to clipboard operation
mage copied to clipboard

Add support for additional `apoc.convert` functions

Open tonijurjevic96 opened this issue 7 months ago • 2 comments

Description

Please briefly explain the changes you made here.

Pull request type

  • [ ] Bugfix
  • [x] Algorithm/Module
  • [ ] Feature
  • [ ] Code style update (formatting, renaming)
  • [ ] Refactoring (no functional changes, no api changes)
  • [ ] Build related changes
  • [ ] Documentation content changes
  • [ ] Other (please describe):

Related issues

Delete if this PR doesn't resolve any issues. Link the issue if it does.

######################################

Reviewer checklist (the reviewer checks this part)

Module/Algorithm

  • [x] Core algorithm/module implementation
  • [x] Query module implementation
  • [x] Tests provided (unit / e2e)
  • [ ] Code documentation
  • [ ] README short description

Documentation checklist

  • [ ] Add the documentation label tag
  • [ ] Add the bug / feature label tag
  • [ ] Add the milestone for which this feature is intended
    • If not known, set for a later milestone
  • [ ] Write a release note, including added/changed clauses
    • [Release note text]
  • [ ] Link the documentation PR here
    • [Documentation PR link]
  • [ ] Tag someone from docs team in the comments

Summary by CodeRabbit

  • New Features
    • Added a procedure to recursively flatten nested lists into a single-level list.
    • Introduced a procedure to convert Memgraph graph elements (nodes, edges, paths) into nested tree-like dictionary structures, with support for property filtering.
    • Added utilities for converting Memgraph and Python types to JSON strings and parsing JSON lists, including support for datetime and timedelta types.
    • Provided procedures to load JSON from local files and remote URLs into Memgraph.
  • Bug Fixes
    • None.
  • Tests
    • Added end-to-end and unit tests for list flattening, graph-to-tree conversion, and JSON utilities, covering a wide range of scenarios and edge cases.
    • Introduced mock modules and fixtures to support isolated testing of graph and JSON utilities.
  • Chores
    • Updated subproject references and added module-level docstrings for clarity.

tonijurjevic96 avatar Jun 10 '25 08:06 tonijurjevic96

Walkthrough

This update introduces new utilities and features for handling graph data structures and JSON serialization in Memgraph-related modules. It adds a recursive list-flattening function to the C++ collections module, new Python procedures for converting graph elements to tree-like dictionaries, and robust JSON serialization/deserialization utilities. Comprehensive unit and end-to-end tests are provided for all new functionalities.

Changes

File(s) Change Summary
cpp/collections_module/CMakeLists.txt Changed module name in build configuration from "collections" to "collections_module".
cpp/collections_module/algorithm/collections.cpp, collections.hpp Added Collections::Flatten function and related constants for flattening nested lists; updated header with new function declaration.
cpp/collections_module/collections_module.cpp Registered the new Flatten function as a procedure in module initialization.
cpp/memgraph Updated subproject commit reference.
e2e/test_collections.py Added end-to-end tests for the new collections_module.flatten procedure, covering various nested list scenarios and graph data.
python/convert.py Added a procedure to convert Memgraph graph elements (nodes, edges, paths) into tree-like dictionaries, with property filtering support.
python/json_util.py Added JSON serialization (to_json) and deserialization (from_json_list) procedures for Memgraph types and general Python data.
python/mage/convert/init.py, python/tests/convert/init.py Added module docstrings for new convert module and its tests.
python/mage/convert/mock_mgp.py Introduced mock classes for graph elements (Label, Type, Vertex, Edge, Path, Record) for testing.
python/mage/convert/test_convert.py Added unit tests for graph-to-tree conversion logic, covering nodes, relationships, paths, and nested/primitive structures.
python/mage/json_util/init.py Added JSON utilities: load from file/URL, serialize graph elements to JSON, and parse JSON lists; includes recursive type conversion.
python/mage/json_util/mock_mgp.py Introduced mock graph and mgp classes for testing JSON utilities.
python/mage/json_util/test_json_util.py Added unit tests for JSON serialization of primitives, graph elements, datetime types, and complex/nested structures.
python/tests/convert/conftest.py Added pytest fixtures for mock vertex, edge, and path objects for convert module tests.
python/tests/convert/test_convert.py Added unit tests using fixtures for node, relationship, path, nested, and primitive value conversions.
python/tests/json_util/init.py Added module docstring for json_util tests.
python/tests/json_util/conftest.py Added comprehensive mock mgp and graph element classes for use in json_util tests.
python/tests/json_util/test_json_util.py Added parameterized and edge-case unit tests for JSON serialization/deserialization of various types and structures.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Memgraph
    participant CollectionsModule
    participant PythonProc

    User->>Memgraph: CALL collections_module.flatten([nested_list])
    Memgraph->>CollectionsModule: Invoke Flatten procedure
    CollectionsModule->>CollectionsModule: Recursively flatten list
    CollectionsModule-->>Memgraph: Return flattened list
    Memgraph-->>User: Return result

    User->>Memgraph: CALL convert.to_tree(value, config)
    Memgraph->>PythonProc: Invoke to_tree procedure
    PythonProc->>PythonProc: Recursively convert graph elements to tree
    PythonProc-->>Memgraph: Return tree dictionary
    Memgraph-->>User: Return result

    User->>Memgraph: CALL json_util.to_json(value)
    Memgraph->>PythonProc: Invoke to_json procedure
    PythonProc->>PythonProc: Convert value to JSON-compatible structure
    PythonProc->>PythonProc: Serialize to JSON string
    PythonProc-->>Memgraph: Return JSON string
    Memgraph-->>User: Return result

    User->>Memgraph: CALL json_util.from_json_list(json_str)
    Memgraph->>PythonProc: Invoke from_json_list procedure
    PythonProc->>PythonProc: Parse JSON string, validate as list
    PythonProc-->>Memgraph: Return list
    Memgraph-->>User: Return result

Poem

🐇
A nest of lists, now flat and neat,
Graphs to trees—what a feat!
JSON flows from nodes and paths,
With tests to check each code that lasts.
From files, URLs, and queries too,
The rabbit brings fresh tools to you!

✨ Finishing Touches
  • [ ] 📝 Generate Docstrings

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

coderabbitai[bot] avatar Jun 10 '25 08:06 coderabbitai[bot]

Quality Gate Failed Quality Gate failed

Failed conditions
32.4% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

sonarqubecloud[bot] avatar Jun 10 '25 08:06 sonarqubecloud[bot]

Deadline to merge: 9th July

Josipmrden avatar Jul 04 '25 12:07 Josipmrden

TODO:

  • we didn't test if the apoc function is mapped to the functions
  • build not passing
  • resolve comments

Josipmrden avatar Jul 07 '25 11:07 Josipmrden

Please retry analysis of this Pull-Request directly on SonarQube Cloud

sonarqubecloud[bot] avatar Jul 21 '25 13:07 sonarqubecloud[bot]