DRAFT: toolsets for mongo, redis, Kfuse tempo trace

[chitender]

trafficstars

toolsets for mongo, redis, Kfuse tempo trace

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

:x: chitenderkumar
:x: chitender
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai[bot]]

Walkthrough

Adds a broad InfraInsights integration: new enhanced toolsets (Elasticsearch, Kafka/Kafka Connect, MongoDB/Atlas, Redis, Kubernetes, Kfuse Tempo), InfraInsights V2 client and bases, prompt/routing templates, docs/examples, dependency updates, guarded Azure/pyodbc imports, toolset loading/logging enhancements, tests, and Dockerfile adjustments.

Changes

Cohort / File(s)	Summary
Top-level exports & loader holmes/plugins/toolsets/__init__.py, holmes/plugins/toolsets/infrainsights/__init__.py, holmes/holmes/plugins/toolsets/infrainsights/__init__.py	Export many InfraInsights toolsets; add config-driven loader with multi-name aliases, enhanced logging, instantiate+configure flow, MCP/YAML fallbacks, env substitution, and inner-tool safety checks.
InfraInsights core (base & client) holmes/plugins/toolsets/infrainsights/base_toolset_v2.py, holmes/plugins/toolsets/infrainsights/infrainsights_client_v2.py	New BaseInfraInsightsToolV2/BaseInfraInsightsToolsetV2 and InfraInsightsClientV2: multi-strategy instance resolution (id/name/prompt), caching, health checks, ES helpers, dataclasses, and detailed error messages.
Enhanced Elasticsearch holmes/plugins/toolsets/infrainsights/enhanced_elasticsearch_toolset.py, holmes/holmes/plugins/toolsets/infrainsights/enhanced_elasticsearch_toolset.py	Add EnhancedElasticsearchToolset and many ES/OpenSearch tools (health, indices, cluster/node/index stats, tasks, thread pools, mappings/settings, hot threads, snapshots); supports configure(...) and prereq health checks.
Enhanced MongoDB (incl. Atlas) holmes/plugins/toolsets/infrainsights/enhanced_mongodb_toolset.py, holmes/plugins/toolsets/__init__.py, holmes/MONGODB_TOOLSET_SETUP.md, holmes/examples/*	Add EnhancedMongoDBToolset (health, DB/list, coll stats, perf, slow queries, replset, ops, security, backups, capacity), example configs/docs, and export MongoDBAtlasToolset.
Enhanced Redis holmes/plugins/toolsets/infrainsights/enhanced_redis_toolset.py, holmes/REDIS_TOOLSET_SETUP.md	Add EnhancedRedisToolset (health, perf, memory, keys, slowlog, connections, replication, persistence, cluster, security, capacity, config); configurable client, prereq check, and inner-tool shape checks.
Kubernetes toolset holmes/plugins/toolsets/infrainsights/comprehensive_kubernetes_toolset.py, infrainsights_kubernetes_example.yaml	Add container-aware handler, logs (single/all containers), logs search, pod describe tools; supports kubeconfig/in-cluster loading and aggregated per-container results.
Kafka & Kafka Connect holmes/plugins/toolsets/infrainsights/enhanced_kafka_toolset.py, holmes/plugins/toolsets/infrainsights/comprehensive_kafka_connect_toolset.py, KAFKA_INSTALLATION_GUIDE.md	Add verbose Kafka topics tool and InfraInsightsKafkaConnectToolset with connector REST management (health, list, details, create/update/delete, status, restart); include installation guidance.
Kfuse Tempo toolset holmes/plugins/toolsets/infrainsights/kfuse_tempo_toolset.py, tests/test_prompt_parser.py	Add KfuseTempoToolset with PromptParser and tools for fetching traces (cluster/service) and AnalyzeTraceRCA; support config/env overrides and unit tests for parser.
Prompts & routing templates holmes/plugins/prompts/_general_instructions_enhanced.jinja2, holmes/plugins/prompts/_infrainsights_routing.jinja2, holmes/plugins/prompts/alert_investigation_with_routing.jinja2	Add enhanced investigation instructions, instance-based routing and parameter-extraction rules, and an alert-investigation routing template enforcing instance parameters.
Configs, examples & env scripts config.example.yaml, infrainsights_toolsets.yaml, holmes/examples/*, holmes/examples/*.sh	Add example InfraInsights configs, Kubernetes manifests, environment setup scripts, and service-specific example YAMLs.
Docs & setup guides INFRAINSIGHTS_TOOLSETS.md, holmes/MONGODB_TOOLSET_SETUP.md, holmes/REDIS_TOOLSET_SETUP.md, KAFKA_INSTALLATION_GUIDE.md	Add comprehensive docs and setup guides: installation, configuration, usage, troubleshooting, and tool references.
Dependencies pyproject.toml, holmes/plugins/toolsets/infrainsights/requirements.txt	Add InfraInsights-related dependencies (elasticsearch, pymongo, redis, kafka-python, orjson, cryptography, prometheus-client, asyncio-redis, etc.) and a dedicated requirements file.
Azure SQL / Azure SDK guarded imports holmes/plugins/toolsets/azure_sql/apis/*, holmes/plugins/toolsets/azure_sql/azure_sql_toolset.py	Make Azure/pyodbc imports optional with AZURE_/PYODBC_AVAILABLE guards; constructors raise ImportError at runtime if SDKs missing; add token-based ODBC helper and query-store helpers.
Dockerfile ODBC handling Dockerfile	Make Microsoft ODBC driver installation conditional on Debian VERSION_ID (11/12); skip optional components on unsupported versions with warnings to continue build.
Tool registration & safety holmes/core/tools_utils/tool_executor.py, holmes/core/tools.py	ToolExecutor: skip & log dict-valued tools during registration. tools.py: allow callable prerequisites (arbitrary types) and add JSON serializer for callable field.
MCP error formatting holmes/plugins/toolsets/mcp/toolset_mcp.py	Normalize exception/detail formatting when initializing MCP server tools.
Toolset YAML / tests tests/core/test_toolset_manager.py, tests/test_prompt_parser.py	Tests: YAMLTool and ToolsetYamlFromConfig conversions and override semantics; prompt-parser unit tests for Kfuse PromptParser parsing functions.
Misc. runtime guards & small fixes holmes/core/supabase_dal.py, holmes/core/toolset_manager.py, others	Add early-return guard in supabase DAL, remove duplicate assignment in toolset manager, and other small defensive changes.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant Core as Holmes Core
  participant Loader as Toolset Loader
  participant TS as InfraInsights Toolset
  participant Client as InfraInsightsClientV2

  User->>Core: start with config (custom_toolsets)
  Core->>Loader: load_toolsets_from_config(config)
  Loader->>TS: instantiate toolset class (by alias/name)
  TS->>TS: if config present -> configure(config)
  TS->>Client: init/update InfraInsightsClientV2
  alt configure success
    TS-->>Loader: return configured toolset
  else configure error
    TS-->>Loader: log error and skip
  end
  Loader-->>Core: register returned toolsets

sequenceDiagram
  autonumber
  actor User
  participant Core as Holmes Core
  participant Tool as Enhanced Tool
  participant TS as Toolset (has client)
  participant Client as InfraInsightsClientV2

  User->>Core: "Get index stats for es-prod"
  Core->>Tool: invoke(params: instance_name)
  Tool->>TS: get_infrainsights_client()
  Tool->>Client: resolve_instance(serviceType, instance_name)
  alt instance resolved
    Tool->>Client: get_elasticsearch_index_stats(instance,...)
    Client-->>Tool: data
    Tool-->>Core: StructuredToolResult SUCCESS
  else resolution/error
    Tool-->>Core: StructuredToolResult ERROR with guidance
  end

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120+ minutes

Areas requiring extra attention:

• InfraInsightsClientV2 resolution logic and ES/OpenSearch client fallbacks.
• Toolset loader aliases, instantiate/configure flow, and safety checks for inner tool shapes.
• Azure/pyodbc guarded imports and ODBC token helper for correctness and security.
• Extensive new tool implementations (Kafka Connect, Kubernetes, Kfuse Tempo) for correctness and error handling.

Possibly related PRs

• robusta-dev/holmesgpt#840 — similar early-return guard in holmes/core/supabase_dal.py; overlaps the same DAL defensive change.
• robusta-dev/holmesgpt#459 — refactors toolset loading/management and may overlap with loader/ToolsetManager changes introduced here.
• robusta-dev/holmesgpt#776 — modifies the MongoDB Atlas/toolset area; likely directly related to the MongoDBAtlasToolset import and usage.

Suggested labels

enhancement

Pre-merge checks and finishing touches

❌ Failed checks (2 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title mentions MongoDB, Redis, and Kfuse Tempo toolsets, which are the primary additions in this PR. However, it's marked as DRAFT and uses informal language ('mongo' instead of 'MongoDB'), and the changeset is far more extensive than just these three toolsets—it includes Elasticsearch, Kafka, Kubernetes, comprehensive documentation, and numerous infrastructure improvements.	Consider revising the title to be more comprehensive and remove DRAFT status, e.g.: 'Add InfraInsights toolsets for multi-instance infrastructure monitoring' or 'Add enhanced toolsets for MongoDB, Redis, Elasticsearch, Kafka, and Kubernetes with Kfuse Tempo integration'.
Description check	❓ Inconclusive	The description is extremely terse ('toolsets for mongo, redis, Kfuse tempo trace') and provides no meaningful context about the changes, objectives, rationale, or implementation details. While it does reference the main components being added, it fails to convey the scope and significance of the extensive infrastructure tooling framework being introduced.	Expand the description to explain what InfraInsights toolsets are, their purpose, supported services, configuration approach, and key features. Include rationale for the design choices and any breaking changes or dependencies introduced.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 88.24% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• [ ] 📝 Generate docstrings

🧪 Generate unit tests (beta)

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}