Import workflow does not work

[dhmw]

Describe the bug

When trying to restore a backup to a new user pool, the majority of user records cannot be imported. Most of the records from the pool I backed up fail with the following message:

[FAILED] Line Number NNNN - The User Record does not set any of the auto verified attributes to verified. (Example: email_verified to true).

I am unable to establish any configuration change in the target cognito pool which has any effect on this behaviour. I've tried almost all combinations of settings relating to sign-in and sign-up options and the import still fails.

More to the point, the import fails when I set up the new cognito pool with exactly the same settings as the pool which was backed up.

To Reproduce

Back up a user pool which contains users with a mix of email_verified attribute values (missing, true and false) and then try to import it to a new user pool.

Expected behavior

I would expect all user records to be imported in as much the same state as in the original pool as possible.

Please complete the following information about the solution:

• [ ] Version: v1.0.5 / HEAD of master at https://github.com/aws-solutions/cognito-user-profiles-export-reference-architecture/tree/a923a3317b46de8dc9f4076ce15b667f89390262
• [ ] Region: Export: us-east-1; Import eu-west-1
• [ ] Was the solution modified from the version published on this repository? Only in metadata.
• [ ] If the answer to the previous question was yes, are the changes available on GitHub? No
• [ ] Have you checked your service quotas for the sevices this solution uses? No - could you please advise if this is relevant to the issue?
• [ ] Were there any errors in the CloudWatch Logs? Yes - see description above.

[dhmw]

Digging a bit further into this - I don't understand the auto verified attribute requirement here.

The import fails for any user with no email_verified attribute, or if the attribute is present and set to false.

However, I can manually change the email_verified attribute to true OR false for any existing user.

It seems to me that the import job could actually import all users first setting email_verified: true and then doing a second pass to reset the values to false where appropriate?

Is it me, or is the import process just broken or incomplete?

[jrgaray27]

Good morning, Thank you for reaching out to us with your question.

It appears that you are using the solution to import a User Pool that contains users who are not verified, meaning they do not have the email_verified attribute set to true.

This is currently not possible to do with the Amazon Cognito service. When creating the user import CSV file to import users to a new User Pool, Cognito requires that "at least one of the auto-verified attributes must be true for each user." This requirement can be found in the Amazon Cognito developer guide, under "Formatting the CSV file" section.

https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-using-import-tool-csv-header.html#cognito-user-pools-using-import-tool-formatting-csv-file

Since the solution imports users using this method, only users with at least one of the auto-verified attributes set to true will be successfully imported to the new User Pool.

Thanks.

[dhmw]

Hi @jrgaray27 I appreciate that this solution mirrors the underlying capabilities of Cognito itself.

However, it does seem that a 1:1 backup:restore can be achieved by first importing all users with email_verified: true and then doing a 2nd pass to reset the values to what they were originally.

Therefore, there is a deficiency in this solution to restore all data as much to its original state as possible.

I'm also fairly surprised this issue has not been raised before - does every other person who adopted this solution have pools filled completely with only verified users?

If AWS (or those supporting AWS resources in public) don't seem to want to address this, it just adds another point to my list of grievances with Cognito and I should better spend my time looking for an alternative and a migration path away from Cognito.

Regards, Doug.

[kroeter]

Hi Doug, Thank you for your comments, we appreciate your interest in improving this solution and will add your recommendation to our feature backlog. Unfortunately, we do not have any feature work dedicated to the roadmap for this solution through the remainder of the the year and so I don't see a path forward for achieving this functionality in the near term outside of the workaround that you mention in your comments above.

Thanks again,

Kyle