mcp icon indicating copy to clipboard operation
mcp copied to clipboard

[Tool Description] Improve Resource Group tool description for better LLM selection

Open alzimmermsft opened this issue 3 months ago • 1 comments

Problem

The current Resource Group tool description lacks the detail and clarity needed for optimal LLM tool selection, resulting in an estimated confidence score of ~0.2-0.3 (below target of ≥0.5).

Current Description Issues:

  • Very brief - only mentions listing and managing without specifics
  • Missing usage guidance (when to use vs when not to use)
  • Limited context about resource group purpose in Azure resource hierarchy
  • Poor parameter explanation
  • Missing specific resource group management operations
  • Formatting issues (missing space after period)
  • No mention of ARM templates, deployments, or governance context

Current Description

Resource group operations - Commands for listing and managing Azure resource groups in your subscriptions.This tool is a hierarchical MCP command router. Sub commands are routed to MCP servers that require specific fields inside the "parameters" object. To invoke a command, set "command" and wrap its args in "parameters". Set "learn=true" to discover available sub commands.

Suggested Improvement

Resource group operations - Commands for listing and managing Azure resource groups that serve as logical containers for organizing and managing related Azure resources within subscriptions. Use this tool when you need to list resource groups, create new resource groups, manage resource group properties and tags, delete resource groups, or work with resource group-level operations like deployments and governance policies. Resource groups provide lifecycle management, access control, and billing organization for Azure resources. Do not use this tool for managing individual resources within groups, subscription-level operations, tenant management, or resource-specific configurations - this tool focuses on resource group container management rather than the resources themselves. This tool is a hierarchical MCP command router where sub-commands are routed to MCP servers that require specific fields inside the "parameters" object. To invoke a command, set "command" and wrap its arguments in "parameters". Set "learn=true" to discover available sub-commands for different resource group management operations. Note that this tool requires appropriate Azure permissions to manage resource groups and will only show resource groups accessible to the authenticated user within their subscription scope.

Acceptance Criteria

  • [ ] Update tool description to include clear usage guidance
  • [ ] Explain resource group logical container and organizational context
  • [ ] Clarify when NOT to use this tool (vs individual resources, subscriptions)
  • [ ] Detail the specific resource group management operations available
  • [ ] Mention lifecycle management and governance capabilities
  • [ ] Fix formatting issues
  • [ ] Achieve confidence score ≥0.5 in tool selection testing

Related: Tool description review for improved LLM selection accuracy

alzimmermsft avatar Sep 03 '25 22:09 alzimmermsft

Copied from https://github.com/Azure/azure-mcp/issues/850

alzimmermsft avatar Sep 03 '25 22:09 alzimmermsft