add : mongodb integration

[vipul-maheshwari]

MongoDB Integration for VDF Import/Export

• Purpose: Introduce utilities for importing and exporting VDF data to and from MongoDB.
• Key Changes:
  • Added .env and .venv entries to .gitignore for environment management.
  • Created mongodb_README.md for documentation on MongoDB usage with VDF.
  • Implemented ExportMongoDB and ImportMongoDB classes for data handling.
  • Enhanced requirements.txt to include pymongo for MongoDB interactions.
  • Updated utility functions to support MongoDB-specific operations and data types.
• Impact: This integration allows seamless data transfer between VDF and MongoDB, enhancing data management capabilities.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Support

• **Purpose: ** Add support for exporting data from MongoDB to the VDF format.
• Key Changes:
  • Implemented the ExportMongoDB class that inherits from the ExportVDB base class.
  • Added functionality to connect to a MongoDB Atlas database, select a collection, and export the data to Parquet files.
  • Implemented a method to flatten nested MongoDB documents and handle various BSON data types.
  • Added support for detecting the vector dimension automatically if not provided.
  • Integrated the exported data into the VDF metadata.
• **Impact: ** This change allows users to export data from MongoDB databases to the VDF format, enabling further processing and analysis of the data.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Functionality

• ****Purpose: ** ** Introduce a new feature to export data from MongoDB to a specified format.
• Key Changes:
  • Added .gitignore entries for environment and testing files.
  • Updated requirements.txt to include pymongo.
  • Created mongodb_export.py for handling MongoDB data export.
  • Implemented argument parsing for MongoDB connection and export parameters.
  • Enhanced error handling for MongoDB connection and collection selection.
• ****Impact: ** ** This addition allows users to seamlessly export data from MongoDB, improving data integration capabilities.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Support

• ******Purpose: ** ** ** Add support for exporting data from MongoDB databases to the VDF format.
• Key Changes:
  • Implemented the ExportMongoDB class that inherits from the ExportVDB base class.
  • Added functionality to connect to a MongoDB Atlas instance, retrieve data from a specified collection, and export it to Parquet files.
  • Implemented logic to handle various BSON data types and flatten nested documents.
  • Added support for detecting the vector dimension automatically if not provided.
  • Integrated the new MongoDB export functionality into the command-line interface.
• ******Impact: ** ** ** This change will allow users to export data from MongoDB databases to the VDF format, enabling them to leverage the VDF ecosystem for vector search, embeddings, and other machine learning tasks.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Functionality

• ********Purpose: ** ** ** ** Add support for exporting data from MongoDB databases to the VDF format.
• Key Changes:
  • Introduced a new ExportMongoDB class that inherits from the base ExportVDB class.
  • Implemented methods to connect to a MongoDB database, fetch data from a specified collection, and export the data to Parquet files.
  • Added support for handling various BSON data types (ObjectId, Binary, Regex, Timestamp, Decimal128, Code) during the flattening process.
  • Integrated the new MongoDB export functionality into the command-line interface.
• ********Impact: ** ** ** ** Users can now export data from MongoDB databases to the VDF format, enabling seamless integration with the VDF ecosystem and downstream applications.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Functionality

• **********Purpose: ** ** ** ** ** Introduces a new feature to export data from MongoDB into a specified format.
• Key Changes:
  • Added .cfg and environment-related entries to .gitignore.
  • Updated requirements.txt to include pymongo.
  • Created mongodb_export.py for handling MongoDB data exports.
  • Implemented argument parsing for MongoDB connection and export parameters.
  • Enhanced utility functions to support MongoDB-specific data handling.
• **********Impact: ** ** ** ** ** This addition allows users to seamlessly export data from MongoDB, enhancing the tool's versatility.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Functionality

• ************Purpose: ** ** ** ** ** ** Introduce functionality to export data from MongoDB to a specified format.
• Key Changes:
  • Added .cfg and environment-related entries to .gitignore.
  • Updated requirements.txt to include pymongo for MongoDB support.
  • Implemented ExportMongoDB class for handling MongoDB data exports.
  • Added command-line argument parsing for MongoDB connection and export parameters.
  • Integrated data flattening and exporting to Parquet format.
• ************Impact: ** ** ** ** ** ** This enhancement allows users to seamlessly export data from MongoDB, improving data integration capabilities.

✨ Generated with love by Kaizen ❤️

Original Description

# Add MongoDB Export Functionality

• **************Purpose: ** ** ** ** ** ** ** Adds the ability to export data from a MongoDB database to the VDF format.
• Key Changes:
  • Added a new ExportMongoDB class that inherits from the ExportVDB base class.
  • Implemented methods to connect to a MongoDB database, fetch data from a specified collection, and export the data to Parquet files.
  • Included support for handling various BSON data types (ObjectId, Binary, Regex, Timestamp, Decimal128, Code) during the flattening process.
  • Added a new mongodb subparser to the command-line interface to allow users to specify MongoDB connection details and export options.
• **************Impact: ** ** ** ** ** ** ** This change will enable users to export data from MongoDB databases to the VDF format, allowing for easier integration with the VDF ecosystem and downstream applications.

✨ Generated with love by Kaizen ❤️

Original Description

- [ ] export script - [ ] import script

---

[!IMPORTANT] Adds MongoDB export functionality with BSON handling and Parquet export in mongodb_export.py.

• MongoDB Export Integration:
  • Adds ExportMongoDB class in mongodb_export.py for exporting data from MongoDB.
  • Implements make_parser() and export_vdb() methods for argument parsing and export logic.
  • Handles BSON type conversions and data flattening in flatten_dict().
  • Exports data to Parquet format with vector dimension detection in get_data().
• Configuration:
  • Adds MONGODB to DBNames in names.py.
  • Updates db_metric_to_standard_metric in util.py to include MongoDB distance metrics.
• Dependencies:
  • Adds pymongo to requirements.txt.

^{This description was created by for 6788f900fc2e64c21ba17d05d2844fab454aa712. It will automatically update as commits are pushed.}