Support Partitioning

[felschr]

Support Cosmos DB partitioning.

There is a beta release right now with basic functionality. There are a few drawbacks, though:

• [ ] partition key property is fixed to "/partition"
• [ ] partition key values are fixed to "user" & "role" respectively
• [ ] IUserLoginStore is not supported at the moment (IdentityUser.Logins)

[felschr]

Dynamic JSON property name for the partition key could be supported via a custom ContractResolver: https://stackoverflow.com/a/44435284/6065945

[Tirmisee]

Could you provide an example configuration? I've test with following code i can create a new user but can't sign-in with _userManager.FindByNameAsync I did something wrong?

services.AddIdentityWithDocumentDBStores<MyIdentityUser, MyIdentityRole>(
            dbOptions => {
                dbOptions.DocumentUrl = "https://localhost:8081";
                dbOptions.DocumentKey = "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
                dbOptions.DatabaseId = "Users-db";
                dbOptions.CollectionId = "userCollection";
                dbOptions.PartitionKeyDefinition = new PartitionKeyDefinition
                {
                    Paths = new Collection<string> { "/partition" }
                };

Can I use userName as Partition key?

"AspNetCore.Identity.DocumentDB" Version="2.0.0-beta4"

[felschr]

The configuration looks fine. Did you use any older version before 2.0.0-beta4?

beta4 broke compatibility with older data sets. But in case you need them I can send you some instructions how to migrate them.

[Tirmisee]

Please give us a correct configuration while creating a new Cosmos DB collection to make it work with AspNetCore.Identity.DocumentDB What should use for Partition key and Unique keys?

[felschr]

At the moment the partition key has to be set to /partition. I'll make this configurable in a future release but for now this is the only option.

[rmarskell]

@FelschR Can you post what to do if we need to migrate from pre-beta4?

[felschr]

This commit essentially contains those breaking changes: 4930fc607a637f9adae5d88e4712bb9d5f883f0a Before beta4 there was a single userMapping type. That caused some conflicts when someone uses the email address as the username as well. So I split it up into userMappingUsername and userMappingEmail.

If you want to migrate your data you'd have to replace all documents with type of userMapping with the 2 of the new document types (or only one depending on if your users have both username & email address set). Also keep in mind that this is only necessary when using partitioning. But I figure because you commented in this particular Github issue you are indeed using it.

The layout of those two new documents is basically the same as before:

{
    "partition": "[email protected]", // email address here
    "id": "userMappingEmail",
    "type": "userMappingEmail",
    "targetId": "f6d8c5f6-1b0a-4eec-bea5-c64dba0f44fd"
}

{
    "partition": "[email protected]", // username here (in this case it's the same as email address)
    "id": "userMappingUsername",
    "type": "userMappingUsername",
    "targetId": "f6d8c5f6-1b0a-4eec-bea5-c64dba0f44fd"
}

[rmarskell]

Hmm. What if I was not using partitioning before and I need to add it to an existing collection of users created in beta2 without partition key set? I'm switching to using shared RU's on my database which requires a partition key to be set for each collection, but it seems like you add extra documents for the partitions (or I'm missing something).

[rmarskell]

Ok, I figured out how to do it by exporting the documents to json and using Sublime Text find & replace regex to add the new partition documents. I'm going to put the steps here in case anyone else might find them useful. Note that this is only useful if adding partitioning later.

1. Export the documents from your user collection using the DocumentDB Data Migration Tool. Be sure to prettify the json!
2. Open the json file in Sublime Text 3.
3. Hit Ctrl+H to open the find and replace panel.
4. Enter the following into the Find box:

   (?s)"id": "([^"]*)",(.*?"normalizedUserName": "([^"]*)",\n.*?\n  })

5. Enter the following into the Replace box:

   "partition": "\1",\n    "id": "user",\2,\n  {\n    "partition": "\3",\n    "id": "userMappingUsername",\n    "type": "userMappingUsername",\n    "targetId": "\1"\n  },\n  {\n    "partition": "\3",\n    "id": "userMappingEmail",\n    "type": "userMappingEmail",\n    "targetId": "\1"\n  }

6. Test that it works by doing a single replacement. It should add two new documents following the document that was "replaced", add a partition property which should have the id of the user, and also change the id to the string "user". Note that you may need to tweak some things if your user documents are in a different format or if the normalizedUserName property is before the id property.
7. If it looks good, undo your test replacement (so you don't do it twice for that one) and do Replace All.
8. Scan the file quickly to make sure everything looks good. You can also try to validate that it's in a correct format.
9. To be super safe, I would create a new collection and try to load it before you delete the old one and recreate it. This will allow you to catch any issues from the CosmosDB side of things. When you create the collection, the partition key should be: /partition
10. You can upload the json file to the collection through the portal (local or Azure). Just go to the data browser and click on the collection name, then select the Upload button in the top bar.
11. When you've checked it, delete the original user collection and recreate it, uploading the new document file as noted above.
12. Back in your code, you just need to update your RegisterDocumentDBStores call in Startup.cs to include the partition key:

    .RegisterDocumentDBStores<ApplicationUser, ApplicationRole>( options => {
        //...Other options...
        options.PartitionKeyDefinition = new PartitionKeyDefinition() {
            Paths = new Collection<string> { "/partition" }
        };
    } )
    

    Don't forget using System.Collections.ObjectModel;
13. And finally, make sure you're using a version of this library >= beta4

Hope this helps someone.