Unable to pass images to Llama 3.2 on Bedrock
Describe the bug When I implement the Nova template (https://github.com/promptfoo/promptfoo/blob/main/examples/amazon-bedrock/nova_multimodal_prompt.json) for Llama 3.2 on bedrock, I encounter the error:
[ERROR] TypeError: message.content.trim is not a function
To Reproduce
Use https://github.com/promptfoo/promptfoo/blob/main/examples/amazon-bedrock/nova_multimodal_prompt.json, but configure the provider to point to a bedrock:<LLAMA ARN>. Pass and reference an {{image}} var.
Expected behavior Base64 image is passed to Llama on bedrock for evaluation
Screenshots
System information:
====================================================================================================
Promptfoo Debug Information
====================================================================================================
{
"version": "0.109.1",
"platform": {
"os": "linux",
"release": "6.11.0-21-generic",
"arch": "x64",
"nodeVersion": "v22.14.0"
},
"env": {},
"configInfo": {
"configExists": false,
"configContent": null
}
}
====================================================================================================
- Promptfoo version: 0.109.1
**Additional context**
@decaffeinatedio have you tried running the example located in examples/amazon-bedrock/promptfooconfig.nova.multimodal.yaml?
When I run that example which uses the prompt you reference, I get
@mldangelo Yep! I have no problems running the example against Nova. But when I use the bedrock Llama provider:
- id: bedrock:arn:aws:bedrock:us-east-1:<<REDACTED>>:inference-profile/us.meta.llama3-2-11b-instruct-v1:0
config:
max_gen_len: 256
temperature: 0.7
top_p: 0.9
I'm encountering that error. When I simplify my prompt to text-only, I get responses back from Llama, so I know it isn't a broader connection/permissions issue.
@mldangelo took a quick look at source and this doesn't seem straightforward to address. Assume it won't be addressed in the short term?