referenceAuth fails when userpool in another region

[constantsphere]

Environment information

System:
  OS: Windows 11 10.0.26100
  CPU: (12) x64 Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
  Memory: 1.87 GB / 15.77 GB
Binaries:
  Node: 22.11.0 - C:\Program Files\nodejs\node.EXE
  Yarn: undefined - undefined
  npm: 10.9.0 - C:\Program Files\nodejs\npm.CMD
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/auth-construct: 1.6.1
  @aws-amplify/backend: 1.14.3
  @aws-amplify/backend-auth: 1.5.1
  @aws-amplify/backend-cli: 1.4.0
  @aws-amplify/backend-data: 1.4.1
  @aws-amplify/backend-deployer: 1.1.13
  @aws-amplify/backend-function: 1.12.3
  @aws-amplify/backend-output-schemas: 1.4.1
  @aws-amplify/backend-output-storage: 1.1.5
  @aws-amplify/backend-secret: 1.2.0
  @aws-amplify/backend-storage: 1.2.6
  @aws-amplify/cli-core: 1.2.0
  @aws-amplify/client-config: 1.5.8
  @aws-amplify/deployed-backend-client: 1.5.2
  @aws-amplify/form-generator: 1.0.3
  @aws-amplify/model-generator: 1.0.13
  @aws-amplify/platform-core: 1.6.5
  @aws-amplify/plugin-types: 1.8.1
  @aws-amplify/sandbox: 1.2.4
  @aws-amplify/schema-generator: 1.2.5
  aws-amplify: 6.9.0
  aws-cdk: 2.1010.0
  aws-cdk-lib: 2.190.0
  typescript: 5.4.5
AWS environment variables:
  AWS_BRANCH = sandbox
No CDK environment variables

Describe the bug

I have a working production Amplify web site hosted in eu-west-1 with the same site also successfully built in us-west-2 by connecting to the same GitHub branch. I would like the us-west-2 site to share the eu-west-1 user pool to avoid the complexities of synchronising users across regions but benefitting for locally run lambda functions. However, when I try to use referenceAuth to reference the user pool I get an error saying it doesn't exist even though I know it does.

I have managed to get referenceAuth to work when referencing a user pool in the same region so it appears to be a cross region issue. It may be related to permissions as I haven't set any special permissions to allow this. Looking at the service role, it doesn't appear to reference anything region specific. In an attempt to simplify the problem I am now trying to get my eu-west-1 sandbox to reference my us-west-2 user pool.

This looks like a bug to me (but may just be a permissions issue?) any help would be very much appreciated.

when trying to access my existing us-west-2 user pool from my eu-west-1 sandbox logs show

Failed resources:
amplify-datamine4-simon-sandbox-3f7f1aa9a7-auth179371D7-LAB767R2IQBQ | 10:09:04 | CREATE_FAILED        | Custom::AmplifyRefAuth     | auth/AmplifyRefAuthCustomResource/Default (AmplifyRefAuthCustomResource) Received response status [FAILED] from custom resource. Message returned: User pool us-west-2_NuOnAiUIa does not exist.

Logs: /aws/lambda/amplify-datamine4-simon-s-AmplifyRefAuthCustomReso-yaB2YNjzIwBr

    at de_ResourceNotFoundExceptionRes (/var/runtime/node_modules/@aws-sdk/client-cognito-identity-provider/dist-cjs/index.js:4736:21)
    at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-cognito-identity-provider/dist-cjs/index.js:4345:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:167:18
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:321:38
    at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async ReferenceAuthInitializer.getUserPool (/var/task/index.js:89:30)
    at async ReferenceAuthInitializer.getResourceDetails (/var/task/index.js:180:50)
    at async ReferenceAuthInitializer.handleEvent (/var/task/index.js:62:135) (RequestId: 96a41d3a-4b12-4009-9386-aba6bed04267)
amplify-datamine4-simon-sandbox-3f7f1aa9a7 | 10:09:21 | UPDATE_FAILED        | AWS::CloudFormation::Stack | auth.NestedStack/auth.NestedStackResource (auth179371D7) Embedded stack arn:aws:cloudformation:eu-west-1:682648571864:stack/amplify-datamine4-simon-sandbox-3f7f1aa9a7-auth179371D7-LAB767R2IQBQ/715f8750-1c76-11f0-8721-02feff9b3ebf was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to create: [AmplifyRefAuthCustomResource]. 
The CloudFormation deployment has failed.
Caused By: The stack named amplify-datamine4-simon-sandbox-3f7f1aa9a7 failed to deploy: UPDATE_ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: User pool us-west-2_NuOnAiUIa does not exist.

Resolution: Find more information in the CloudFormation AWS Console for this stack.

amplify/auth/resource.ts looks like this...

import { defineAuth, referenceAuth } from '@aws-amplify/backend';
import { userMigration } from './migrate-users/resource';
import { postConfirmation } from './registerUser/resource';
import { resetPassword } from './reset-password/resource';

const isPrimaryRegion = ( process.env.AWS_REGION == 'us-west-2' );
let tempAuth;

if( isPrimaryRegion ) {
		
	/**
	 * need to ensure the ALLOW_USER_PASSWORD_AUTH Authentication flows is allowed under:
	 * Amazon Cognito > User pools > amplifyAuthUserPool > App clients > App client: amplifyAuthUserPoolAppClient 
	 */
	tempAuth = defineAuth({
		loginWith: { email: {
		verificationEmailStyle: "CODE",
		verificationEmailSubject: "ConstantGraph confirmation code",
		verificationEmailBody: (createCode) => `<html>...cut for simplicity...</html>`,
		}, },
		triggers: { userMigration, postConfirmation },
		userAttributes: { "custom:MongoUserId": { dataType: "String", mutable: true } },
		access: (allow) => [
			allow.resource( resetPassword ).to( ["deleteUser"] ),
			allow.resource( postConfirmation ).to( ["manageUsers"] ),
		  ],
	});

} else {

	// referenceAuth doesn't seem to work across regions
	console.log(`creating ref auth with userPoolId: ${process.env.userPoolId!}, userPoolClientId: ${process.env.userPoolClientId!}, identityPoolId: ${process.env.identityPoolId!}, authRoleArn: ${process.env.authRoleArn!}, unauthRoleArn: ${process.env.unauthRoleArn!}`);
	tempAuth = referenceAuth({
		userPoolId: process.env.userPoolId!,
		userPoolClientId: process.env.userPoolClientId!,
		identityPoolId: process.env.identityPoolId!,
		authRoleArn: process.env.authRoleArn!,
		unauthRoleArn: process.env.unauthRoleArn!
   });

}

export const auth = tempAuth;

Reproduction steps

1. create a simple Amplify website with authentication in us-west-2 region and deploy.
2. create another Amplify website using referenceAuth in eu-west-1 region. The values for referenceAuth are all copied directly from the relevant places in the us-west-2 region AWS Cognito console.
3. run npx amplify sandbox in eu-west-1.

[AnilMaktala]

Hey @constantsphere, Thank you for bringing this to our attention. Currently, the referenceAuth feature does not support the use of cross-region Cognito user pools. We have logged this as a feature request for the team.

[constantsphere]

@AnilMaktala thank you very much for investigating. Do you have any workarounds for this situation with a global / multi-region website? I've looked through various forums and can't seem to find this specific issue anywhere. Could you perhaps suggest a link to somewhere I could ask? Many thanks!