AI Kit / AIConversation with Amazon Nova Micro not working in EU due to Cross-region inference

[jgo80]

Environment information

System:
  OS: macOS 15.3.2
  CPU: (10) arm64 Apple M1 Max
  Memory: 996.53 MB / 32.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 23.7.0 - /opt/homebrew/bin/node
  Yarn: 4.7.0 - /opt/homebrew/bin/yarn
  npm: 10.9.2 - /opt/homebrew/bin/npm
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/ai-constructs: 1.3.0
  @aws-amplify/auth-construct: 1.6.1
  @aws-amplify/backend: 1.14.3
  @aws-amplify/backend-ai: Not Found
  @aws-amplify/backend-auth: 1.5.1
  @aws-amplify/backend-cli: 1.5.0
  @aws-amplify/backend-data: 1.4.1
  @aws-amplify/backend-deployer: 1.1.20
  @aws-amplify/backend-function: 1.12.3
  @aws-amplify/backend-output-schemas: 1.4.1
  @aws-amplify/backend-output-storage: 1.1.5
  @aws-amplify/backend-secret: 1.2.0
  @aws-amplify/backend-storage: 1.2.6
  @aws-amplify/cli-core: 1.4.1
  @aws-amplify/client-config: 1.5.8
  @aws-amplify/data-construct: 1.15.1
  @aws-amplify/data-schema: 1.19.0
  @aws-amplify/deployed-backend-client: 1.5.2
  @aws-amplify/form-generator: 1.0.5
  @aws-amplify/model-generator: 1.0.13
  @aws-amplify/platform-core: 1.6.5
  @aws-amplify/plugin-types: 1.8.1
  @aws-amplify/sandbox: 1.2.12
  @aws-amplify/schema-generator: 1.2.8
  aws-amplify: 6.13.5
  aws-cdk: 2.1004.0
  aws-cdk-lib: 2.184.1
  typescript: 5.8.2
No AWS environment variables
No CDK environment variables

Describe the bug

Working with AI Kit / AIConversation with Amazon Nova Micro from eu-central-1 leads to this error:

Invocation of model ID amazon.nova-micro-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

Reproduction steps

I am working from eu-central-1

Here is my Setup:

import { ClientSchema, a, defineData } from '@aws-amplify/backend';

const schema = a.schema({
  chat: a
    .conversation({
      aiModel: a.ai.model('Amazon Nova Micro'),
      systemPrompt: 'You are a helpful assistant.',
      inferenceConfiguration: {}
    })
    .authorization((allow) => allow.owner()),
});

export type Schema = ClientSchema<typeof schema>;

export const data = defineData({
  schema,
  authorizationModes: {
    defaultAuthorizationMode: 'iam',
  },
});

The client connects, but no conversation can be started. The WebSocket Stream contains this error:

Invocation of model ID amazon.nova-micro-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

I guess this is because within the EU, the endpoint needs to bee eu.amazon.nova-micro-v1:0 according to Cross-region inference.

Docs state, that Amazon Nova Micro is supported.

[ykethan]

Hey,👋 thanks for raising this! I'm going to transfer this over to our API repository for better assistance 🙂

[AnilMaktala]

Hey @jgo80, Thank you for bringing this up. Could you please try the workaround suggested here: https://github.com/aws-amplify/docs/issues/8121#issuecomment-2494375015 and let me know if it resolves your issue?

[jgo80]

@AnilMaktala thanks for your reply. I totally understand the workaround.

As the word implies, it is only a (unfortunately rather unattractive) workaround.

My expectation is, that the amplify backend uses the correct crossRegionModel based on the deployment region. Otherwise the whole Amplify Gen2 approach makes no sense, then I can just use plain CDK.

So again: thanks for the temporary workaround, but I would like to see this work out of the box. This is no feature request, this is what developers would expect. Please keep this issue open until resolved. Thank you 🤗

[AnilMaktala]

Hey @jgo80, Thank you for sharing your feedback — I fully understand your perspective. We already have a feature request #3076 open for this. Feel free to add your input or comments there.