Migration from v3 to V4

[jt-pt-dev]

Question

I am struggling a little with the migration from v3 to v4. I understand consulting is a paid for option and that is fair, but will a migration doc be released at some point?

My solution followed the Quickstart docs pretty closely for v3. Now with v4 I have changed ApiResources to ApiScopes (as I was getting an 'invalid_scope' error) but now I get 'Audience validation error' when the MVC app calls the Api. Database migrations were run based on RockSolidKnowledge/IdentityServer4.Migration.Scripts but only needed the new columns in DeviceCodes and PersistedGrants as Api, Clients and Identity resources are run in memory.

[StefanKoenigMUC]

would also be interested for a migration guide.

[brockallen]

but now I get 'Audience validation error'

https://identityserver4.readthedocs.io/en/latest/topics/resources.html#apis

[leastprivilege]

OK - I pinned this issue.

Please report all migration issues you run into. This can be then used to create a guide at some point. This is a community effort.

[jt-pt-dev]

Thank you Brock and Dominick for your replies. I have my solution working now. I will try and summarise my experience in case it helps others or in case I am "doing it wrong" and others can learn.

My system has multiple clients (currently ASP.NET Core web applications and in the future mobile apps). They are related apps in a safety management system for mining/construction/engineering scenarios. Each client basically has its own API. They are relatively simple and small in size. IdentityServer is used for authentication for each app and API and I am using ASPNET Identity. It very similar to the Quickstart examples.

I keep Identity resources, API resources and Clients in Config.cs and keep the Operational store in SQL Server. I do this as the resources and clients will change very infrequently and all apps and APIs and IdentityServer will live on the one server.

The relevant part of my v3 Config.cs was like the docs:

public IEnumerable<ApiResource> Apis =>
	new ApiResource[]
	{
		new ApiResource(configuration["ApiNames:MyActionsApi"], "MyActions API")
	};

The relevant part of my v3 Startup.cs was:

var builder = services.AddIdentityServer()
			.AddInMemoryIdentityResources(config.Ids)
			.AddInMemoryApiResources(config.Apis)
			.AddInMemoryClients(config.Clients)
			.AddOperationalStore(options =>
			{
				options.ConfigureDbContext = b => b.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"));
				options.EnableTokenCleanup = true;
			})
			.AddAspNetIdentity<ApplicationUser>();

I upgraded the Nuget packages to v4 and ran the appropriate migrations scripts from https://github.com/RockSolidKnowledge/IdentityServer4.Migration.Scripts/tree/CreateScripts for my database.

The first problem I had was IdentityServer returned an 'invalid_scope' error. After reading the docs I changed my Config.cs Apis from ApiResource to ApiScope. This solved the 'invalid_scope' error but authentication at the API was failing with 'Audience validation error'. This occurs because "When using the scope-only model, no aud (audience) claim will be added to the token". (https://identityserver4.readthedocs.io/en/latest/topics/resources.html#apis)

I then changed to include BOTH ApiScope and ApiResource and this solved the problem. I used the same value for Scope name and Resource name. I don't know if this is the "right" solution but it works so far for my simple project.

The relevant part of my v4 Config.cs:

public IEnumerable<ApiScope> ApiScopes =>
	new ApiScope[]
	{
		new ApiScope(configuration["ApiNames:MyActionsApi"], "MyActions API")
	};

public IEnumerable<ApiResource> ApiResources =>
	new ApiResource[]
	{
		new ApiResource(configuration["ApiNames:MyActionsApi"], "MyActions API")
		{
			Scopes = { configuration["ApiNames:MyActionsApi"] }
		}
	};

The relevant part of my v4 Startup.cs:

var builder = services.AddIdentityServer()
			.AddInMemoryIdentityResources(config.Ids)
			.AddInMemoryApiScopes(config.ApiScopes)
			.AddInMemoryApiResources(config.ApiResources)
			.AddInMemoryClients(config.Clients)
			.AddOperationalStore(options =>
			{
				options.ConfigureDbContext = b => b.UseSqlServer(Configuration.GetConnectionString("DefaultConnection"));
				options.EnableTokenCleanup = true;
			})
			.AddAspNetIdentity<ApplicationUser>();

Hope that helps and I am not leading people in the wrong direction.

[bryantlikes]

I have a fairly simple API so I just switched out the ApiResource for ApiScope and it worked fine. The update blurb was helpful but it would be nice if it was more visible instead of hidden at the bottom of that page.

I also ran into an issue where the ClientId property was removed from the AuthorizationRequest model and replaced with Client (which has the missing ClientId property) so that was a fairly easy fix. I create a PR for the documentation which still has ClientId in it even though it isn't there.

Everything else seems to be working fine.

[zyofeng]

One of the issue I am encountering, is that version 4.0 is checking Content-Type header for "application/x-www-form-urlencoded"

I'm calling token endpoint from Axios in React Native 0.62 and there are some compatiblity issues with Flipper causing Content-Type header to be stripped out. I understand it's not specifically a bug but it would be nice if there is a way to disable HasApplicationFormContentType checking. Additionally I have legacy client apps deployed without using Content-Type header which prevents upgrade to version 4.

[hallidev]

Few issues I've run into that I'm unsure how to handle:

1:

Current broken code:

await _interaction.GrantConsentAsync(context, ConsentResponse.Denied);

ConsentResponse.Denied no longer exists, but I'm unsure how to construct a ConsentResponse that means Denied. The old implementation of Denied looked like this:

public static ConsentResponse Denied = new ConsentResponse();

Is an empty ConsentResponse still the equivalent of Denied?

2:

Current broken code:

await HttpContext.SignInAsync(user.Id, user.UserName, props);

I notice that there is now an overload that takes IdentityServerUser, but I'm unfamiliar with this class and how to get an instance of it. I see that there's a constructor that takes sub. Is it enough to just do var user = new IdentityServerUser(user.Id) and pass that to SignInAsync?

3:

Current broken code:

await _events.RaiseAsync(new ConsentGrantedEvent(User.GetSubjectId(), request.Client.ClientId, request.ScopesRequested, grantedConsent.ScopesValuesConsented, grantedConsent.RememberConsent));

There used to be a ScopesRequested property on AuthorizationRequest. I'm not sure what the intended replacement is.

4: Above, @jt-pt-dev links to some SQL scripts for updating the database, but are there Entity Framework migration files available? If so, how do we use them?

[brockallen]

ConsentResponse.Denied no longer exists, but I'm unsure how to construct a ConsentResponse that means Denied.

There's a new DenyConsent API.

[hallidev]

@brockallen Gotcha - I will take a look at that. Are there simple answers for my other questions? Was just hoping to get some guidance on the breaking changes.

[srinijss]

I am getting the error "Unable to resolve service for type - 'IdentityServer4.Services.IReplayCache' when i run my IdentityServer project after upgrading IdentityServer4 nuget package from 3.1.0 to 4.0.1. Please let me know if you need more details on this. Appreciate any help on this. Would also be interested for a migration guide. Thanks.

[mikegoatly]

@hallidev I can help with this question:

2:

Current broken code:

await HttpContext.SignInAsync(user.Id, user.UserName, props);

I notice that there is now an overload that takes IdentityServerUser, but I'm unfamiliar with this class and how to get an instance of it. I see that there's a constructor that takes sub. Is it enough to just do var user = new IdentityServerUser(user.Id) and pass that to SignInAsync?

If you look at the updated quickstart sample you can see an example of how to construct IdentityServerUser here.

I suspect you'll be able to answer some of your other questions by looking through the updated quickstart code as well, but I've not had to cross those particular bridges 😄

[Omotayad]

Hi, I am currently facing this issue after migration, whenever I call the token endpoint. Am i missing a library or configuration?

An exception was thrown while activating IdentityServer4.Endpoints.TokenEndpoint -> IdentityServer4.Validation.ClientSecretValidator -> IdentityServer4.Validation.SecretValidator -> λ:IdentityServer4.Validation.ISecretValidator[] -> IdentityServer4.Validation.PrivateKeyJwtSecretValidator."

[leastprivilege]

https://www.identityserver.com/articles/migrating-your-identityserver4-v3-database-to-identityserver4-v4

[jinweijie]

Finished deploying the migration to production yesterday.

The overall migration experience was smooth, it took around 2 days to finish preparing the upgrade script, code changes, test and deployment. Thanks to the migration script: https://github.com/RockSolidKnowledge/IdentityServer4.Migration.Scripts

One gotcha was the PublicOrigin property was removed in v4, so we didn't find the issue util we deployed to dev environment (We use Nginx as reverse proxy). Then follow this instruction to replace the PublicOrigin with SetIdentityServerOrigin method in Configure method of Startup.cs: https://github.com/IdentityServer/IdentityServer4/issues/4535 (Please be aware that place SetIdentityServerOrigin above UseIdentityServer method)

Thanks for all the resources and hope this also helps a bit.

[jinweijie]

Few issues I've run into that I'm unsure how to handle:

1:

Current broken code:

await _interaction.GrantConsentAsync(context, ConsentResponse.Denied);

ConsentResponse.Denied no longer exists, but I'm unsure how to construct a ConsentResponse that means Denied. The old implementation of Denied looked like this:

public static ConsentResponse Denied = new ConsentResponse();

Is an empty ConsentResponse still the equivalent of Denied?

2:

Current broken code:

await HttpContext.SignInAsync(user.Id, user.UserName, props);

I notice that there is now an overload that takes IdentityServerUser, but I'm unfamiliar with this class and how to get an instance of it. I see that there's a constructor that takes sub. Is it enough to just do var user = new IdentityServerUser(user.Id) and pass that to SignInAsync?

3:

Current broken code:

await _events.RaiseAsync(new ConsentGrantedEvent(User.GetSubjectId(), request.Client.ClientId, request.ScopesRequested, grantedConsent.ScopesValuesConsented, grantedConsent.RememberConsent));

There used to be a ScopesRequested property on AuthorizationRequest. I'm not sure what the intended replacement is.

4: Above, @jt-pt-dev links to some SQL scripts for updating the database, but are there Entity Framework migration files available? If so, how do we use them?

for Q1 maybe:

grantedConsent = new ConsentResponse { Error = AuthorizationError.AccessDenied };

for Q3: maybe:

await _events.RaiseAsync(new ConsentDeniedEvent(User.GetSubjectId(), request.Client.ClientId, request.ValidatedResources.RawScopeValues));

[MCFHTAGENTS]

Hi, I am using the PublicOrigin option in my call to AddIdentityServer. Can someone tell me where that has gone in version 4?

also in my

                Mock<IIdentityServerInteractionService> mockInterationService = new Mock<IIdentityServerInteractionService>();
                mockInterationService.Setup(a => a.GetAuthorizationContextAsync(It.IsAny<string>())).Returns(Task.FromResult(new AuthorizationRequest()
                {
                    ScopesRequested = dummyScope,
                }));

I get the ScopeRequested failing what should this call be replaced with?

Thank you

[wangfengjun404]

@MCFHTAGENTS

two ways to fix it

https://github.com/IdentityServer/IdentityServer4/issues/4631

app.Use(async (context, next) =>
            {
                //升级到v4后这publicorigin选项不见了，改为在中间件中设置publicorigin
                var config = context.RequestServices.ResolveConfig_();
                var publicOrigin = config["public_origin"];
                if (publicOrigin?.Length > 0)
                {
                    context.SetIdentityServerOrigin(publicOrigin);
                    context.SetIdentityServerBasePath(context.Request.PathBase.Value.TrimEnd('/'));
                }
                await next.Invoke();
            });

[MCFHTAGENTS]

Thank you - has anyone any idea on my other point above?

I have also now moved forward to testing the new system and get the following error in my logs (and this all used to work on the old version of IdentityServer4):

2020-07-19 09:47:16.692 +01:00 [INF] Invoking IdentityServer endpoint: IdentityServer4.Endpoints.AuthorizeEndpoint for /connect/authorize
2020-07-19 09:47:16.693 +01:00 [DBG] Start authorize request
2020-07-19 09:47:16.695 +01:00 [DBG] No user present in authorize request
2020-07-19 09:47:16.697 +01:00 [DBG] Start authorize request protocol validation
2020-07-19 09:47:16.780 +01:00 [DBG] [redacted email address] found in database: true
2020-07-19 09:47:16.780 +01:00 [VRB] Calling into client configuration validator: IdentityServer4.Validation.DefaultClientConfigurationValidator
2020-07-19 09:47:16.783 +01:00 [DBG] client configuration validation for client [redacted email address] succeeded.
2020-07-19 09:47:16.791 +01:00 [DBG] Checking for PKCE parameters
2020-07-19 09:47:16.791 +01:00 [DBG] No PKCE used.
2020-07-19 09:47:16.828 +01:00 [DBG] Found ["openid"] identity scopes in database
2020-07-19 09:47:16.872 +01:00 [DBG] Found [] API resources in database
2020-07-19 09:47:16.887 +01:00 [DBG] Found ["[redacted scope name]"] scopes in database
2020-07-19 09:47:16.909 +01:00 [ERR] Scope [redacted scope name] not found in store.
2020-07-19 09:47:16.911 +01:00 [ERR] Request validation failed

I can see an entry with [redacted scope name] in the following tables in my database: ApiScopes, ClientScopes but NOT IdentityResources

I can see in the code this error arising from a failed call to: resourcesFromStore.FindApiScope(requestedScope.ParsedName);

but it's there - any thoughts and ideas given the changes?

[MCFHTAGENTS]

OK - one update on the error above - I needed to set Enabled to true in the ApiScope definition

[ishchatul]

For anyone that has implemented their own ISecretsListValidator the params the Validate async method receives have been switched: now - Task<SecretValidationResult> ValidateAsync(IEnumerable<Secret> secrets, ParsedSecret parsedSecret); used to be - Task<SecretValidationResult> ValidateAsync(ParsedSecret parsedSecret, IEnumerable<Secret> secrets);

[kikaragyozov]

#4742

[seven1986]

I have add ApiResources 'books' and ApiResourceScopes 'books.read' and ApiScope 'books.read'

[HuntJason]

I just came across the article Learn how to migrate from your IdentityServer4 v3 database to IdentityServer4 v4 through manual migrations and EntityFramework Migrations. Hope someone finds this helpful.

[snothub]

One of my migration woes is that we have one legacy .net framework app that gets the id_token using form_post mechanism and then validates that id_token and nonce within using /identitytokenvalidation endpoint. That endpoint is no more in v4, so I'm left wondering what I should be doing instead. Any suggestions @leastprivilege ?

[leastprivilege]

This is not exactly a v3 to v4 issue, right? This endpoint is gone since a couple of years now.

Either way - introspection is the replacement for it.

[snothub]

Either way - introspection is the replacement for it.

What credentials should be used for validating id_token? This is all identity resources and those have no secrets.

[leastprivilege]

I see. Well you need a JWT library in your clients s then.

[w0rt4]

@srinijss you need to add extra line to your Identity Server configuration. As showed in this link you need to execute extention (AddJwtBearerClientAuthentication()) method on your identityserverbuilder object to register default implementation of IReplayCache

I am getting the error "Unable to resolve service for type - 'IdentityServer4.Services.IReplayCache' when i run my IdentityServer project after upgrading IdentityServer4 nuget package from 3.1.0 to 4.0.1. Please let me know if you need more details on this. Appreciate any help on this. Would also be interested for a migration guide. Thanks.

[leastprivilege]

Since the workaround for the removed PublicOrigin seems to come up again and again - here's the solution

https://github.com/IdentityServer/IdentityServer4/issues/4857#issuecomment-689719505

[IndexOverflow]

I'm having some issues in my ProfileService after updating to v4. When the ProfileDataRequestContext.Caller is for an access_token ResourceValidationResult contains the expected ApiResources. However, when the caller is for an identity_token the validation result is empty; no resources and the parsed scopes list is also empty. As far as I can tell the refactor regarding ApiResource and ApiScopes shouldn't affect IdentityResources (which again, works as expected after a minor refactor).