kubectl-ai icon indicating copy to clipboard operation
kubectl-ai copied to clipboard

MCP Client Misbehave

Open tuannvm opened this issue 6 months ago • 8 comments

After the latest refactoring of the tools package, it looks like the MCP client is having some issues with converting the schema.

λ kubectl-ai % ./kubectl-ai -v=4 --llm-provider openai --model gpt-4.1 --mcp-client
E0621 08:49:24.433555   91552 manager.go:118] connecting to MCP server "permiflow": initializing connection: initializing MCP client: transport error: failed to send request: Post "http://localhost:8080/mcp": dial tcp [::1]:8080: connect: connection refused
E0621 08:49:29.507575   91552 main.go:335] Failed to initialize MCP client: MCP tool registration failed: operation failed after 3 attempts: listing tools from MCP server "sequential-thinking": parsing tools from MCP server: converting MCP input schema to tool input schema: converting MCP input schema to tool input schema: unexpected input schema type "integer" for key "revisesThought": map[description:Which thought is being reconsidered minimum:1 type:integer]

tuannvm avatar Jun 21 '25 15:06 tuannvm

Also, managing schema differences between multiple MCP servers have room for improvement.

λ kubectl-ai % ./kubectl-ai -v=4 --llm-provider openai --model gpt-4.1 --mcp-client

  MCP Server Status:                                                          



  Successfully connected to 3 MCP server(s) (4 tools discovered)              



                                                                              
    • sequential-thinking (npx) - Connected, Tools: sequentialthinking        



                                                                              
    • resend (node) - Connected, Tools: send-email                            



                                                                              
    • cloudflare-documentation (remote) - Connected, Tools:                   
  search_cloudflare_documentation, migrate_pages_to_workers_guide             



  Hey there, what can I help you with today?                                  


>>> show namespace
E0621 08:57:31.977211   93981 openai.go:454] Error in OpenAI streaming: POST "https://api.openai.com/v1/chat/completions": 400 Bad Request {
    "message": "Invalid schema for function 'migrate_pages_to_workers_guide': In context=(), object schema missing properties.",
    "type": "invalid_request_error",
    "param": "tools[2].function.parameters",
    "code": "invalid_function_parameters"
  }
E0621 08:57:31.977313   93981 conversation.go:191] "error reading streaming LLM response" err=<
        OpenAI streaming error: POST "https://api.openai.com/v1/chat/completions": 400 Bad Request {
            "message": "Invalid schema for function 'migrate_pages_to_workers_guide': In context=(), object schema missing properties.",
            "type": "invalid_request_error",
            "param": "tools[2].function.parameters",
            "code": "invalid_function_parameters"
          }
 >
Error: reading streaming LLM response: OpenAI streaming error: POST "https://api.openai.com/v1/chat/completions": 400 Bad Request {
    "message": "Invalid schema for function 'migrate_pages_to_workers_guide': In context=(), object schema missing properties.",
    "type": "invalid_request_error",
    "param": "tools[2].function.parameters",
    "code": "invalid_function_parameters"

tuannvm avatar Jun 21 '25 15:06 tuannvm

https://github.com/GoogleCloudPlatform/kubectl-ai/pull/375 cc @justinsb @droot

tuannvm avatar Jun 21 '25 16:06 tuannvm

@tuannvm As i have checked, your changes have fixed the sequentialthinking tool error but I am getting below errors for another mcp server(awslabs.cost-explorer-mcp-server):

E0623 10:29:53.690409 7836 main.go:335] Failed to initialize MCP client: MCP tool registration failed: operation failed after 3 attempts: listing tools from MCP server "awslabs.cost-explorer-mcp-server": parsing tools from MCP server: converting MCP input schema to tool input schema: converting MCP input schema to tool input schema: expected type for key "date_range": map[$ref:#/$defs/DateRange]

This has also fixed the issue mentioned in https://github.com/GoogleCloudPlatform/kubectl-ai/issues/353 but encountered above issue while using mcp server(awslabs.cost-explorer-mcp-server)

vaib-2310 avatar Jun 23 '25 10:06 vaib-2310

Thanks, @vaib-2310, for the feedback. Let me think of a better solution for the input schema. Looks like we need to have a catch-all condition.

tuannvm avatar Jun 24 '25 04:06 tuannvm

https://github.com/GoogleCloudPlatform/kubectl-ai/blob/main/pkg/mcp/client.go#L274-L283

I would like to check @droot / @justinsb opinions before implementing

		// Fallback: treat any unrecognized schema as generic object
		klog.V(2).InfoS("Unrecognized schema format, treating as object", "key", key)
		gollmSchema.Type = gollm.TypeObject
		return gollmSchema, nil
	}

Would something like this sufficient? Treat all non-native schemaType as object

tuannvm avatar Jun 24 '25 05:06 tuannvm

Would something like this sufficient? Treat all non-native schemaType as object

That sounds reasonable to me.

droot avatar Jun 24 '25 21:06 droot

thanks @droot @vaib-2310 please try again if the issue has been fixed for you

tuannvm avatar Jun 25 '25 19:06 tuannvm

thanks @droot @vaib-2310 please try again if the issue has been fixed for you

@tuannvm , I have asked 2 questions , for the 1st question I got an error, while for the 2nd one it had answered me correctly, looks like some more tuning it require:

Below is the output:

1st Question:

Show me the monthly amortized cost for EC2 and its associated resources tagged Role:Openshift4 and User:xxx.xxx which are of instance type m7i.2xlarge in May 2025 and also show which resources took how much cost.

Running: [MCP: awslabs.cost-explorer-mcp-server] get_cost_and_usage(date_range=map[end_date:2025-05-31 start_date:2025-05-01], filter_expression=map[And:[map[Dimensions:map[Key:SERVICE MatchOptions:[EQUALS] Values:[Amazon Elastic Compute Cloud - Compute]]] map[Tags:map[Key:Role MatchOptions:[EQUALS] Values:[Openshift4]]] map[Tags:map[Key:User MatchOptions:[EQUALS] Values:[xxx.xxx]]] map[Dimensions:map[Key:INSTANCE_TYPE MatchOptions:[EQUALS] Values:[m7i.2xlarge]]]]], granularity=MONTHLY, group_by=[map[Key:RESOURCE_ID Type:DIMENSION]], metric=AmortizedCost)

Do you want to proceed ?

  1. Yes
  2. Yes, and don't ask me again
  3. No Enter your choice (1,2,3): 1 E0626 10:51:18.323917 9211 conversation.go:341] "error executing action" err=< calling MCP tool "get_cost_and_usage" on server "awslabs.cost-explorer-mcp-server": tool returned an error: &{Result:{Meta:map[]} Content:[{Annotated:{Annotations:} Type:text Text:Error executing tool get_cost_and_usage: 2 validation errors for get_cost_and_usageArguments group_by.dict[str,str] Input should be a valid dictionary [type=dict_type, input_value=[{'Key': 'RESOURCE_ID', 'Type': 'DIMENSION'}], input_type=list] For further information visit https://errors.pydantic.dev/2.11/v/dict_type group_by.str Input should be a valid string [type=string_type, input_value=[{'Key': 'RESOURCE_ID', 'Type': 'DIMENSION'}], input_type=list] For further information visit https://errors.pydantic.dev/2.11/v/string_type}] IsError:true}

output=null Error: executing action: calling MCP tool "get_cost_and_usage" on server "awslabs.cost-explorer-mcp-server": tool returned an error: &{Result:{Meta:map[]} Content:[{Annotated:{Annotations:} Type:text Text:Error executing tool get_cost_and_usage: 2 validation errors for get_cost_and_usageArguments group_by.dict[str,str] Input should be a valid dictionary [type=dict_type, input_value=[{'Key': 'RESOURCE_ID', 'Type': 'DIMENSION'}], input_type=list] For further information visit https://errors.pydantic.dev/2.11/v/dict_type group_by.str Input should be a valid string [type=string_type, input_value=[{'Key': 'RESOURCE_ID', 'Type': 'DIMENSION'}], input_type=list] For further information visit https://errors.pydantic.dev/2.11/v/string_type}] IsError:true}

2nd Question:

Get monthly costs for EC2 services tagged Role:Openshift4 and User:xxx.xxx in eu-west-1 for May 2025

Running: [MCP: awslabs.cost-explorer-mcp-server] get_cost_and_usage(date_range=map[end_date:2025-05-31 start_date:2025-05-01], filter_expression=map[And:[map[Dimensions:map[Key:SERVICE MatchOptions:[EQUALS] Values:[Amazon Elastic Compute Cloud - Compute]]] map[Tags:map[Key:Role MatchOptions:[EQUALS] Values:[Openshift4]]] map[Tags:map[Key:User MatchOptions:[EQUALS] Values:[xxx.xxx]]] map[Dimensions:map[Key:REGION MatchOptions:[EQUALS] Values:[eu-west-1]]]]], granularity=MONTHLY, metric=UnblendedCost)

Do you want to proceed ?

  1. Yes
  2. Yes, and don't ask me again
  3. No Enter your choice (1,2,3): 1

The total unblended cost for Amazon Elastic Compute Cloud - Compute in eu- west- 1 for the month of May 2025, for resources tagged with Role:Openshift4 and User:xxx.xxx, was $xxxx

vaib-2310 avatar Jun 26 '25 11:06 vaib-2310