[Question]: Integration with Amazon bedrock issue

[malphi]

Checklist

• [X] I've searched for similar issues and couldn't find anything matching
• [X] I've included steps to reproduce the behavior

Affected Components

• [X] K8sGPT (CLI)
• [ ] K8sGPT Operator

K8sGPT Version

0.3.29

Kubernetes Version

v1.29

Host OS and its Version

MacOS

Steps to reproduce

1. I try to add aws bedrock as provider, follow this wiki to setup: https://docs.k8sgpt.ai/reference/providers/backend/#amazon-bedrock command is : k8sgpt auth add --backend amazonbedrock --model ai21.j2-ultra-v1 --providerRegion us-east-1
2. I setup ENV, add permission in aws IAM, and used a command (aws bedrock list-foundation-models) to test, I can get model list.
3. Error: but, when I use 'k8sgpt analyze -e -b amazonbedrock' to run, I got a error: Error: failed while calling AI provider amazonbedrock: AccessDeniedException: You don't have access to the model with the specified model ID..
4. Based on source code , seems like k8sgpt for now only support Claude model?

Expected behaviour

integration with bedrock success.

Actual behaviour

Error: failed while calling AI provider amazonbedrock: AccessDeniedException: You don't have access to the model with the specified model ID.

Additional Information

No response

[AlexsJones]

Yes that's right, though it's possible to adjust this, we just haven't tested. If I get time next week I'll look at this for you.

[awsyshah]

@AlexsJones

You ok for me to attempt this ? Making the request from the 'right' account :).

Thanks,

[awsyshah]

@malphi - I was trying to reproduce the error , so while we cannot use the other models apart from the anthropic.claude-v1/2 ones , I don't get the error as if one of the other models is selected the code defaults to anthropic.claude-v2.

And based on the error , may be you don't have access the "anthropic.claude-v2" model where you were testing.

The request for including other models is still valid , just that the response would not be an error , but a attempt at claude.

[awsyshah]

@AlexsJones - Have made an attempt and submitted a pull https://github.com/k8sgpt-ai/k8sgpt/pull/1099 not too sure about the process so looking for guidance on the missing bits , added a signature on new pushes with no changes but intermediate pushes still have a error.

We should not need additional documentation as the documents don't outline which models are supported specifically, so current instructions work.

There are next steps to add all the other models, but did not want to make too many additions on a single pr, can add other models as a follow-up.

[AlexsJones]

Thanks @awsyshah ill take a look at that

[awsyshah]

@AlexsJones cleaned it up and submitted a fresh PR.

[malphi]

@malphi - I was trying to reproduce the error , so while we cannot use the other models apart from the anthropic.claude-v1/2 ones , I don't get the error as if one of the other models is selected the code defaults to anthropic.claude-v2.

And based on the error , may be you don't have access the "anthropic.claude-v2" model where you were testing.

The request for including other models is still valid , just that the response would not be an error , but a attempt at claude.

Yes, It works when I fix model permission issue, Thanks.

[malphi]

@AlexsJones - Have made an attempt and submitted a pull #1099 not too sure about the process so looking for guidance on the missing bits , added a signature on new pushes with no changes but intermediate pushes still have a error.

We should not need additional documentation as the documents don't outline which models are supported specifically, so current instructions work.

There are next steps to add all the other models, but did not want to make too many additions on a single pr, can add other models as a follow-up.

Thanks for your support. I went through the PR, my suggestion is, can you provide more flexible way to add models such as configuration? Otherwise we have to change code if the AI provider add new models.

[awsyshah]

Thanks for your support. I went through the PR, my suggestion is, can you provide more flexible way to add models such as configuration? Otherwise we have to change code if the AI provider add new models.

I agree, it would be good to be able to pass the models as a parameter, but the nature of the models with different request & response formats , different max limits. my thinking is it will still need some significant rework , potentially a local conf file for the user to define the format of the request and response , which would be a total revamp of the current solution approach.

I might suggest we extend the client for now with the current approach and can look at a wider PR to completely parameterize the model input.

@AlexsJones thoughts ?

[AlexsJones]

I hear what you're saying. But by restricting the models we support, we are intentionally putting some experience guard rails up. We know that they each take slightly different parameterisation. I think the small compromise here is needing a PR to support more, but it keeps the quality of the experience high for everyone.

[awsyshah]

@AlexsJones Let me know your thoughts on the PR if it looks good now with the modifications.