EntityFramework.Docs icon indicating copy to clipboard operation
EntityFramework.Docs copied to clipboard

Scaling SaaS model (one DbContext, database-per-tenant) with EF Core?

Open jeremy-holovacs-sp opened this issue 3 years ago • 7 comments

There doesn't seem to be any good guidance on how to set up and maintain a database-per-tenant model for EF Core.

  • How to create/ migrate new tenant dbs programmatically?
  • How to manage connection strings?
  • How to select a specific tenant at runtime?

Sharding seemed to be supported in .NET 4.x but no support in .NET Core or beyond. How is EF Core used in a scalable, sustainable manner in a SaaS environment with tenant data isolation?


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

jeremy-holovacs-sp avatar May 17 '22 18:05 jeremy-holovacs-sp

Sharding seemed to be supported in .NET 4.x

Can you elaborate what you're referring to here?

roji avatar May 17 '22 19:05 roji

Sure, there's support for elastic tooling in .NET 4.x (see here) which seems to support programmatically scaling database "shards" in Azure, but is not supported for .NET Core or .NET 5.x/ 6.x+ (see the issues page)

The project also does not seem to be receiving any official support at all at this point, which makes very few options available for programmatically scaling out a SaaS database-per-tenant model.

jeremy-holovacs-sp avatar May 17 '22 19:05 jeremy-holovacs-sp

@jeremy-holovacs-sp that seems to be an Azure SQL-specific library, that specifically works at the ADO.NET layer, and so probably isn't relevant for working with EF Core. It indeed doesn't seem to have received much attention recently (last commit in 2020), but that project doesn't really have anything to do with us...

Regardless, our docs do contain some information on the database-per-tenant model - see here and below - which isn't to say there isn't room to add more. All that isn't to say we shouldn't look into improving our guidance for using EF Core with database sharding

roji avatar May 17 '22 19:05 roji

@roji That's understood; I was offering the Azure project as an example of a dynamic, scalable SaaS database-per-tenant model that I feel is tragically underrepresented for EF; if the project were still actively supported we'd probably have been able to use that instead, but it's not. We use EF Core against Azure, but are running into serious scaling challenges because of this limitation and we're having to get... creative... in our horizontal scaling efforts. It's very frustrating.

I have seen the documentation you referred to, but that really doesn't solve the main problem we have. We need to dynamically add databases, maintain a store of connection strings, and programmatically run migrations at scale. The example documentation is very static, and requires storing connection string data for a finite number of databases in a configuration file, which is... messy, among other things. It has to be maintained at design-time.

I'd really like a more enterprise-ready scaling solution. I would hope to see some guidance/ support for this in the future.

jeremy-holovacs-sp avatar May 17 '22 19:05 jeremy-holovacs-sp

@jeremy-holovacs-sp I understand.

First, the building blocks are generally there: you can tell EF Core to dynamically create database at runtime, apply migrations to them, etc. If you simply want to have database-per-tenant, you can simply generate a unique connection string per tenant (exactly how is database-specific!), and do the creation/migration the first time, etc.

Now, I understand that's a lot to do yourself, and it would indeed be nice if EF Core just managed everything for you. Since there's probably quite a range of tweaks and possibilities in sharding, this may be more appropriate implemented in high-level layer on top of EF Core. We could also definitely improve our doc guidance on this, to help users do their own thing.

Note also @JonPSmith's library and blog series which helps in exactly this kind of scenario - you may want to check it out.

roji avatar May 17 '22 20:05 roji

Hi @jeremy-holovacs-sp

I too found the elastic tooling code, which has a great set of features, but isn't supported any more. Version 3 of my AuthPermissions.AspNetCore library provides both straight sharing and a hybrid version, just like elastic tooling (did) do. I should say that some elastic tooling split-merge features, like stopping access to a tenant while it is moved, aren't in yet but I have some ideas on how to do that.

As well as the articles that @roji mentioned you might like to see the ASP.NET Core community standup I did a a month ago about Multi-Tenant applications. This gives you a overview of the features (and my approach) to multi-tenant - and as Jon Galloway said, there a lot of other libraries that help with multi-tenant apps.

JonPSmith avatar May 18 '22 09:05 JonPSmith

@JonPSmith That's fantastic, I'll take a look at this. Thanks!

jeremy-holovacs-sp avatar May 18 '22 11:05 jeremy-holovacs-sp