Crash on "Request and Install Certificate" (App Service Environment with custom domain)

[jon-walton]

Hi,

version: 0.9.6

When trying to request a cert in an app service running within an app service environment, I get the following crash.

This happens whether or not letsencrypt:AzureDefaultWebSiteDomainName is configured. When letsencrypt:AzureDefaultWebSiteDomainName is configured, the specified domain is no longer in the hosts listbox during installation.

Thanks

[ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: startIndex]
   System.String.Insert(Int32 startIndex, String value) +7032952
   LetsEncrypt.Azure.Core.KuduHelper.MakeScmUri(String defaultHostName, IAzureWebAppEnvironment settings) in D:\a\1\s\LetsEncrypt.SiteExtension.Core\KuduHelper.cs:27
   LetsEncrypt.Azure.Core.KuduHelper.GetKuduClient(WebSiteManagementClient client, IAzureWebAppEnvironment settings) in D:\a\1\s\LetsEncrypt.SiteExtension.Core\KuduHelper.cs:11
   LetsEncrypt.Azure.Core.Services.<GetKuduRestClient>d__9.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\Services\KuduFileSystemAuthorizationChallengeProvider.cs:77
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.Azure.Core.Services.<WriteFile>d__7.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\Services\KuduFileSystemAuthorizationChallengeProvider.cs:59
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.Azure.Core.Services.<EnsureWebConfig>d__5.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\Services\KuduFileSystemAuthorizationChallengeProvider.cs:43
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.Azure.Core.Services.<Authorize>d__5.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\Services\BaseHttpAuthorizationChallengeProvider.cs:57
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.Azure.Core.Services.<RequestCertificate>d__5.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\Services\AcmeService.cs:44
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.Azure.Core.<RequestInternalAsync>d__16.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\CertificateManager.cs:231
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.Azure.Core.<RequestAndInstallInternalAsync>d__17.MoveNext() in D:\a\1\s\LetsEncrypt.SiteExtension.Core\CertificateManager.cs:244
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   LetsEncrypt.SiteExtension.Controllers.<Install>d__7.MoveNext() in D:\a\1\s\LetsEncrypt-SiteExtension\Controllers\HomeController.cs:250
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
   System.Web.Mvc.Async.TaskAsyncActionDescriptor.EndExecute(IAsyncResult asyncResult) +92
   System.Web.Mvc.Async.<>c__DisplayClass8_0.<BeginInvokeAsynchronousActionMethod>b__1(IAsyncResult asyncResult) +22
   System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult) +42
   System.Web.Mvc.Async.<>c__DisplayClass11_0.<InvokeActionMethodFilterAsynchronouslyRecursive>b__0() +80
   System.Web.Mvc.Async.<>c__DisplayClass11_2.<InvokeActionMethodFilterAsynchronouslyRecursive>b__2() +387
   System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult) +42
   System.Web.Mvc.Async.<>c__DisplayClass3_6.<BeginInvokeAction>b__4() +42
   System.Web.Mvc.Async.<>c__DisplayClass3_1.<BeginInvokeAction>b__1(IAsyncResult asyncResult) +188
   System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult) +38
   System.Web.Mvc.<>c.<BeginExecuteCore>b__152_1(IAsyncResult asyncResult, ExecuteCoreState innerState) +26
   System.Web.Mvc.Async.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult) +73
   System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult) +52
   System.Web.Mvc.Async.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult) +39
   System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult) +38
   System.Web.Mvc.<>c.<BeginProcessRequest>b__20_1(IAsyncResult asyncResult, ProcessRequestState innerState) +40
   System.Web.Mvc.Async.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult) +73
   System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) +38
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +648
   System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) +213
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +131

[dawolf82]

I have the same issue.

[dawolf82]

Did you ever find a resolution?

[jon-walton]

No, we stopped trying. Instead, we put a proxy in front of the app service environment

[ohadschn]

The relevant code is here: https://github.com/sjkp/letsencrypt-siteextension/blob/master/LetsEncrypt.SiteExtension.Core/KuduHelper.cs

The function is pretty simple:

public static Uri MakeScmUri(string defaultHostName, IAzureWebAppEnvironment settings)
{
    var i = defaultHostName.IndexOf("." + settings.AzureWebSitesDefaultDomainName);
    return new Uri($"https://{defaultHostName.Insert(i, ".scm")}");
}

Basically it's looking for foo.azurewebsites.net and replaces it with foo.scm.azurewebsites.net. The stack trace suggests that your app service's default domain doesn't include the default domain name (so i == -1 and the exception ensues).

So either change the default domain name or AzureDefaultWebSiteDomainName so that the substring is matched. Alternatively, you can install the WebJob I wrote on top of this extension's logic which I believe doesn't hit this code path: https://github.com/ohadschn/letsencrypt-webapp-renewer

[dawolf82]

Thanks for directing me to the webapp-renewer project you wrote @ohadschn . After trying it, I get the same error however...

OhadSoft.AzureLetsEncrypt.Renewal.WebJob.AppSettings.AppSettingsRenewer.<Renew>d__4.MoveNext() in C:\projects\letsencrypt-webapp-renewer\src\OhadSoft.AzureLetsEncrypt.Renewal.WebJob\AppSettings\AppSettingsRenewer.cs:line 34<--- [10/08/2020 19:53:15 > a31680: INFO] [10/08/2020 19:53:15 > a31680: ERR ] [10/08/2020 19:53:15 > a31680: ERR ] Unhandled Exception: System.AggregateException: Encountered exception(s) during cert renewal (and/or notification) ---> System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. [10/08/2020 19:53:15 > a31680: ERR ] Parameter name: startIndex [10/08/2020 19:53:15 > a31680: ERR ] at System.String.Insert(Int32 startIndex, String value) [10/08/2020 19:53:15 > a31680: ERR ] at LetsEncrypt.Azure.Core.KuduHelper.MakeScmUri(String defaultHostName, IAzureWebAppEnvironment settings) [10/08/2020 19:53:15 > a31680: ERR ] at LetsEncrypt.Azure.Core.KuduHelper.GetKuduClient(WebSiteManagementClient client, IAzureWebAppEnvironment settings) [10/08/2020 19:53:15 > a31680: ERR ] at LetsEncrypt.Azure.Core.Services.KuduFileSystemAuthorizationChallengeProvider.<GetKuduRestClient>d__9.MoveNext()

Any suggestions on how I can get around this?

Thanks, Danny

[ohadschn]

I guess my extension hits this code path too then. Are you running on a sovereign cloud (Fairfax, Mooncake, or BlackForest)? If so you'll need to set the configuration as such: https://github.com/sjkp/letsencrypt-siteextension/wiki/Azure-Germany,-US-or-China (corresponding section in my docs: https://github.com/ohadschn/letsencrypt-webapp-renewer#sovereign-cloud-mooncake-blackforest-etc).

If not, first double check that you access your website via <app-name>.azurewebsites.net, where <app-name> is the name of your app. And while you're at it, make sure you can access <app-name>.scm.azurewebsites.net.

Then if further debugging is still necessary, I would explicitly set azureDefaultWebSiteDomainName to azurewebsites.net and attach the full log from my webjob (scrub personal information if necessary).

[IsQiao]

China:

letsencrypt:AzureAuthenticationEndpoint

• https://login.windows.net/
• https://login.chinacloudapi.cn/

letsencrypt:AzureTokenAudience

• https://management.core.windows.net/
• https://management.core.chinacloudapi.cn/

letsencrypt:AzureManagementEndpoint

• https://management.azure.com/
• https://management.chinacloudapi.cn

letsencrypt:AzureDefaultWebSiteDomainName

• azurewebsites.net
• chinacloudsites.cn