Feat add Amazon Bedrock support

[usamimeri]

User description

Features #1174

• add support for Amazon Bedrock Provider (Meta, Mistral, Amazon, Ai21, Anthropic, Cohere)
• add test for Amazon Bedrock

config

llm:
  api_type: "bedrock" 
  model: "meta.llama3-70b-instruct-v1:0"  # see metagpt/provider/bedrock/utils.py
  region_name: "REGION"
  access_key: "YOUR_ACCESS_KEY"
  secret_key: "YOUR_SECRET_KEY"

test result

platform win32 -- Python 3.9.19, pytest-8.1.1, pluggy-1.5.0
rootdir: D:\python_projects\MetaGPT
plugins: anyio-4.3.0, asyncio-0.23.6, cov-5.0.0, html-4.1.1, metadata-3.1.1, mock-3.14.0, timeout-2.3.1, xdist-3.5.0
asyncio: mode=strict
collected 129 items

tests\metagpt\provider\postprocess\test_base_postprocess_plugin.py .                                                                                                        [  0%]
tests\metagpt\provider\postprocess\test_llm_output_postprocess.py .                                                                                                         [  1%]
tests\metagpt\provider\test_amazon_bedrock_api.py ............................................................................................                              [ 72%]
tests\metagpt\provider\test_anthropic_api.py .                                                                                                                              [ 73%]
tests\metagpt\provider\test_azure_llm.py .                                                                                                                                  [ 74%]
tests\metagpt\provider\test_base_llm.py ..                                                                                                                                  [ 75%]
tests\metagpt\provider\test_dashscope_api.py .                                                                                                                              [ 76%]
tests\metagpt\provider\test_general_api_base.py .......                                                                                                                     [ 82%]
tests\metagpt\provider\test_general_api_requestor.py ...                                                                                                                    [ 84%]
tests\metagpt\provider\test_google_gemini_api.py .                                                                                                                          [ 85%]
tests\metagpt\provider\test_human_provider.py .                                                                                                                             [ 86%]
tests\metagpt\provider\test_metagpt_llm.py .                                                                                                                                [ 86%]
tests\metagpt\provider\test_ollama_api.py .                                                                                                                                 [ 87%]
tests\metagpt\provider\test_openai.py ........                                                                                                                              [ 93%]
tests\metagpt\provider\test_qianfan_api.py .                                                                                                                                [ 94%]
tests\metagpt\provider\test_spark_api.py ...                                                                                                                                [ 96%]
tests\metagpt\provider\test_zhipuai_api.py ..                                                                                                                               [ 98%]
tests\metagpt\provider\zhipuai\test_async_sse_client.py .                                                                                                                   [ 99%]
tests\metagpt\provider\zhipuai\test_zhipu_model_api.py .                                                                                                                    [100%]

======================================================================== 129 passed, 7 warnings in 10.59s ========================================================================

completion test

models = ["cohere.command-text-v14", "mistral.mixtral-8x7b-instruct-v0:1", "meta.llama3-70b-instruct-v1:0",
              "anthropic.claude-3-haiku-20240307-v1:0", "ai21.j2-grande-instruct", "amazon.titan-text-express-v1"]
    for model_id in models:
        print(f"============{model_id}=============")
        my_config.model = model_id
        api = AmazonBedrockLLM(my_config)
        msg = [{"role": 'user', "content": "Please say: I am \{YOUR_MODEL_NAME\}"}]
        print(api.completion(msg))
        api._chat_completion_stream(msg)

2024-04-27 13:13:15.799 | INFO     | metagpt.const:get_metagpt_package_root:29 - Package root set to D:\python_projects\MetaGPT
============cohere.command-text-v14=============
 Hello, World!
 Hello, World!
============mistral.mixtral-8x7b-instruct-v0:1=============
 Hello World! I'm a helpful, respectful, and honest assistant, always ready to provide accurate and appropriate information. How can I assist you today? Is there a specific question or topic you'd like to discuss? I'm here to help make your life easier and more enjoyable.
 Hello World! I'm a helpful, respectful, and honest assistant, always ready to provide accurate and appropriate information. How can I assist you today? Is there a specific question or topic you'd like to discuss? I'm here to help make your life easier and more enjoyable.
============meta.llama3-70b-instruct-v1:0=============
Hello World!
Hello World!
============anthropic.claude-3-haiku-20240307-v1:0=============
Hello World!
Hello World!
============ai21.j2-grande-instruct=============
assistant: Hello World
============amazon.titan-text-express-v1=============
Sorry, but this model is not capable of saying "Hello World".
Sorry, but this model is not capable of saying "Hello World".

[codiumai-pr-agent-pro[bot]]

PR Description updated to latest commit (https://github.com/geekan/MetaGPT/commit/4c77d6c4547af8266828dbb601d184d2bf31a8cb)

• [ ] Copy walkthrough table to "Files Changed" Tab

[codiumai-pr-agent-pro[bot]]

PR Review

⏱️ Estimated effort to review [1-5]	4, because the PR introduces a significant amount of new code across multiple files, including new classes, methods, and configurations specific to the Amazon Bedrock API integration. The complexity of the code, especially with the handling of asynchronous operations and streaming responses, requires a thorough understanding of both the existing system architecture and the new API's capabilities.
🧪 Relevant tests	Yes
🔍 Possible issues	Possible Bug: The code assumes that all responses from the Amazon Bedrock API will successfully convert to JSON without handling potential exceptions that might occur during the parsing.
	Performance Concern: The synchronous calls to the Amazon Bedrock API might lead to blocking I/O operations that could affect the performance of the entire application, especially under high load.
🔒 Security concerns	No

---

✨ Review tool usage guide:

---

Overview: The review tool scans the PR code changes, and generates a PR review which includes several types of feedbacks, such as possible PR issues, security threats and relevant test in the PR. More feedbacks can be added by configuring the tool.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.

• When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

• With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

See the review usage page for a comprehensive guide on using this tool.

[codiumai-pr-agent-pro[bot]]

PR Code Suggestions

Category	Suggestions
Maintainability	Improve variable naming for clarity. Consider using a more descriptive variable name than resp_cont_tmpl for better code readability. The current name does not clearly indicate the purpose or content of the template. tests/metagpt/provider/req_resp_const.py [69] message=ChatCompletionMessage( - role="assistant", content=resp_cont_tmpl.format(name=name)), + role="assistant", content=response_content_template.format(name=name)),
	Reduce code duplication by using a factory function for object creation. Refactor the repeated creation of CompletionUsage objects with identical parameters into a shared function or a factory pattern to reduce code duplication and improve maintainability. tests/metagpt/provider/req_resp_const.py [80-81] -usage=CompletionUsage(completion_tokens=110, - prompt_tokens=92, total_tokens=202) +def create_default_completion_usage(): + return CompletionUsage(completion_tokens=110, + prompt_tokens=92, total_tokens=202) +usage = create_default_completion_usage()
Possible issue	Ensure variable is defined before use to prevent runtime errors. To avoid potential runtime errors, ensure that resp_cont_tmpl is defined and accessible within the scope where it is used, or pass it explicitly to functions that require it. tests/metagpt/provider/req_resp_const.py [69] +# Ensure resp_cont_tmpl is defined earlier in the code or passed as a parameter message=ChatCompletionMessage( role="assistant", content=resp_cont_tmpl.format(name=name)),
	Add validation for empty or None prompt text in get_request_body method. The method get_request_body in the AmazonProvider class should handle the case where messages_to_prompt returns None or an empty string, which could lead to invalid JSON structure. It's recommended to add a check and raise an appropriate exception if necessary. metagpt/provider/bedrock/bedrock_provider.py [69-72] +prompt_text = self.messages_to_prompt(messages) +if not prompt_text: + raise ValueError("Prompt text is empty or None") body = json.dumps({ - "inputText": self.messages_to_prompt(messages), + "inputText": prompt_text, "textGenerationConfig": generate_kwargs })
Best practice	Use constants or configuration for token values to enhance maintainability. Instead of hardcoding token values directly in the function, consider defining them as constants or retrieving them from a configuration file to make the code more flexible and maintainable. tests/metagpt/provider/req_resp_const.py [73-74] -usage=CompletionUsage(completion_tokens=110, - prompt_tokens=92, total_tokens=202), +# Define token values as constants or retrieve from a config +DEFAULT_COMPLETION_TOKENS = 110 +DEFAULT_PROMPT_TOKENS = 92 +DEFAULT_TOTAL_TOKENS = 202 +usage=CompletionUsage(completion_tokens=DEFAULT_COMPLETION_TOKENS, + prompt_tokens=DEFAULT_PROMPT_TOKENS, total_tokens=DEFAULT_TOTAL_TOKENS),
	Prevent key conflicts in generate_kwargs in get_request_body of AnthropicProvider. The get_request_body method in the AnthropicProvider class should ensure that the generate_kwargs does not contain keys that might conflict with the hardcoded keys in the dictionary, potentially leading to unexpected behavior. metagpt/provider/bedrock/bedrock_provider.py [21-23] +reserved_keys = {"messages", "anthropic_version"} +if any(key in generate_kwargs for key in reserved_keys): + raise KeyError("generate_kwargs contains keys that are reserved: " + ", ".join(reserved_keys)) body = json.dumps( {"messages": messages, "anthropic_version": "bedrock-2023-05-31", **generate_kwargs})
Enhancement	Dynamically compute total_tokens to ensure accuracy. To ensure the total_tokens value is always accurate, compute it dynamically based on the sum of completion_tokens and prompt_tokens rather than hardcoding it. tests/metagpt/provider/req_resp_const.py [73-74] -usage=CompletionUsage(completion_tokens=110, - prompt_tokens=92, total_tokens=202), +completion_tokens = 110 +prompt_tokens = 92 +usage=CompletionUsage(completion_tokens=completion_tokens, + prompt_tokens=prompt_tokens, total_tokens=completion_tokens + prompt_tokens),
	Validate llama_version input in the constructor of MetaProvider. In the MetaProvider class, the constructor should validate the llama_version argument to ensure it only receives 'llama2' or 'llama3' as valid inputs. metagpt/provider/bedrock/bedrock_provider.py [41-43] def __init__(self, llama_version: Literal["llama2", "llama3"]) -> None: + if llama_version not in ["llama2", "llama3"]: + raise ValueError("Invalid llama_version, must be 'llama2' or 'llama3'") self.llama_version = llama_version
Error handling	Add error handling for JSON parsing in get_choice_text_from_stream. The get_choice_text_from_stream method in the AmazonProvider class should handle potential JSON parsing errors with a try-except block to prevent crashes due to malformed data. metagpt/provider/bedrock/bedrock_provider.py [78-81] -rsp_dict = json.loads(event["chunk"]["bytes"]) -completions = rsp_dict["outputText"] +try: + rsp_dict = json.loads(event["chunk"]["bytes"]) + completions = rsp_dict["outputText"] +except json.JSONDecodeError: + raise ValueError("Failed to decode JSON from stream")
	Add key existence validation in get_choice_text_from_stream. The get_choice_text_from_stream method in the BaseBedrockProvider class should validate the structure of the rsp_dict before accessing nested keys to avoid KeyError. metagpt/provider/bedrock/bedrock_provider.py [22-25] rsp_dict = json.loads(event["chunk"]["bytes"]) +if "outputText" not in rsp_dict: + raise KeyError("Expected key 'outputText' not found in response dictionary") completions = self._get_completion_from_dict(rsp_dict)

---

✨ Improve tool usage guide:

---

Overview: The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.

• When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

• With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

See the improve usage page for a comprehensive guide on using this tool.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 38.38863% with 130 lines in your changes are missing coverage. Please review.

Project coverage is 33.64%. Comparing base (f201b2f) to head (98cb452).

Files	Patch %	Lines
metagpt/provider/bedrock/amazon_bedrock_api.py	37.03%	51 Missing :warning:
metagpt/provider/bedrock/utils.py	16.27%	36 Missing :warning:
metagpt/provider/bedrock/bedrock_provider.py	46.87%	34 Missing :warning:
metagpt/provider/bedrock/base_provider.py	52.63%	9 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1231       +/-   ##
===========================================
- Coverage   70.22%   33.64%   -36.59%     
===========================================
  Files         316      320        +4     
  Lines       18860    19070      +210     
===========================================
- Hits        13245     6416     -6829     
- Misses       5615    12654     +7039     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[geekan]

well done!

[better629]

lgtm