Umbraco-CMS icon indicating copy to clipboard operation
Umbraco-CMS copied to clipboard

Random RegexMatchTimeoutException

Open madanl1232 opened this issue 3 months ago • 10 comments

Which Umbraco version are you using?

13.10.0

Bug summary

We're hosting on Umbraco Cloud and haven’t deployed any recent releases, yet some frontend pages unexpectedly started returning 500 errors.

Upon checking the Umbraco logs, we found a RegexMatchTimeoutException (details below). Interestingly, the same version and content work perfectly in our dev, staging, and local environments—only production was affected. Even more strangely, the issue resolved itself without any intervention on our part.

We initially raised a ticket with Umbraco Cloud support, but were advised to log the issue here for further investigation.

Umbraco error log: System.Text.RegularExpressions.RegexMatchTimeoutException: The Regex engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors. at System.Text.RegularExpressions.RegexRunner.<CheckTimeout>g__ThrowRegexTimeout|25_0() at System.Text.RegularExpressions.RegexInterpreter.TryMatchAtCurrentPosition(ReadOnlySpan1 inputSpan) at System.Text.RegularExpressions.RegexInterpreter.Scan(ReadOnlySpan1 text) at System.Text.RegularExpressions.Regex.ScanInternal(RegexRunnerMode mode, Boolean reuseMatchObject, String input, Int32 beginning, RegexRunner runner, ReadOnlySpan1 span, Boolean returnNullIfReuseMatchObject) at System.Text.RegularExpressions.Regex.RunAllMatchesWithCallback[TState](String inputString, ReadOnlySpan1 inputSpan, Int32 startat, TState& state, MatchCallback1 callback, RegexRunnerMode mode, Boolean reuseMatchObject) at System.Text.RegularExpressions.RegexReplacement.ReplaceNonSimpleText(Regex regex, String input, Int32 count, Int32 startat) at System.Text.RegularExpressions.Regex.Replace(String input, String replacement) at Microsoft.AspNetCore.Razor.Language.HtmlConventions.ToHtmlCase(String name) at Microsoft.CodeAnalysis.Razor.DefaultTagHelperDescriptorFactory.ConfigureBoundAttribute(BoundAttributeDescriptorBuilder builder, IPropertySymbol property, INamedTypeSymbol containingType) at Microsoft.CodeAnalysis.Razor.DefaultTagHelperDescriptorFactory.<>c__DisplayClass18_1.<AddBoundAttributes>b__0(BoundAttributeDescriptorBuilder attributeBuilder) at Microsoft.AspNetCore.Razor.Language.DefaultTagHelperDescriptorBuilder.BindAttribute(Action1 configure) at Microsoft.CodeAnalysis.Razor.DefaultTagHelperDescriptorFactory.AddBoundAttributes(INamedTypeSymbol type, TagHelperDescriptorBuilder builder) at Microsoft.CodeAnalysis.Razor.DefaultTagHelperDescriptorFactory.CreateDescriptor(INamedTypeSymbol type) at Microsoft.CodeAnalysis.Razor.DefaultTagHelperDescriptorProvider.Execute(TagHelperDescriptorProviderContext context) at Microsoft.CodeAnalysis.Razor.CompilationTagHelperFeature.GetDescriptors() at Microsoft.AspNetCore.Razor.Language.DefaultRazorTagHelperBinderPhase.ExecuteCore(RazorCodeDocument codeDocument) at Microsoft.AspNetCore.Razor.Language.RazorEnginePhaseBase.Execute(RazorCodeDocument codeDocument) at Microsoft.AspNetCore.Razor.Language.DefaultRazorEngine.Process(RazorCodeDocument document) at Microsoft.AspNetCore.Razor.Language.DefaultRazorProjectEngine.ProcessCore(RazorCodeDocument codeDocument) at Microsoft.AspNetCore.Razor.Language.RazorProjectEngine.Process(RazorProjectItem projectItem) at Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation.RuntimeViewCompiler.CompileAndEmit(String relativePath) at Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation.RuntimeViewCompiler.OnCacheMiss(String normalizedPath) --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.CreateCacheResult(HashSet1 expirationTokens, String relativePath, Boolean isMainPage) at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.OnCacheMiss(ViewLocationExpanderContext expanderContext, ViewLocationCacheKey cacheKey) at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.LocatePageFromViewLocations(ActionContext actionContext, String pageName, Boolean isMainPage) at Umbraco.Cms.Web.Website.ViewEngines.ProfilingViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage) at Umbraco.Cms.Web.Common.Controllers.RenderController.CurrentTemplate[T](T model) at lambda_method821(Closure, Object, Object[]) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ExceptionContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Umbraco.Cms.Web.Common.Middleware.BasicAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Umbraco.Cms.Web.BackOffice.Middleware.BackOfficeExternalLoginProviderErrorMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Umbraco.Deploy.Infrastructure.Middleware.BearerTokenAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Site.Core.Composers.ApiRateLimiterComposer.<>c__DisplayClass0_0.<<Compose>b__4>d.MoveNext() in C:\home\site\repository\src\Site.Core\Composers\ApiRateLimitComposer.cs:line 82 --- End of stack trace from previous location --- at Skybrud.Umbraco.Redirects.Middleware.RedirectsMiddleware.InvokeAsync(HttpContext context) at StackExchange.Profiling.MiniProfilerMiddleware.Invoke(HttpContext context) in C:\projects\dotnet\src\MiniProfiler.AspNetCore\MiniProfilerMiddleware.cs:line 112 at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Umbraco.Cms.Web.Common.Middleware.PreviewAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Umbraco.Cms.Web.Common.Middleware.UmbracoRequestLoggingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Site.Core.Middleware.XmlSitemapMiddleware.InvokeAsync(HttpContext context) in C:\home\site\repository\src\Site.Core\Middleware\XmlSitemapMiddleware.cs:line 26 at Site.Core.Middleware.RobotsTxtMiddleware.InvokeAsync(HttpContext context) in C:\home\site\repository\src\Site.Core\Middleware\RobotsTxtMiddleware.cs:line 21 at Our.Umbraco.MaintenanceMode.Middleware.MaintenanceRedirectMiddleware.InvokeAsync(HttpContext context, IBackofficeUserAccessor backofficeUserAccessor) at SixLabors.ImageSharp.Web.Middleware.ImageSharpMiddleware.Invoke(HttpContext httpContext, Boolean retry) at Umbraco.Forms.Web.HttpModules.ProtectFormUploadRequestsMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at Umbraco.Commerce.Cms.Web.Mvc.UmbracoCommerceRequestBufferingMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.InterfaceMiddlewareBinder.<>c__DisplayClass2_0.<<CreateMiddleware>b__0>d.MoveNext() --- End of stack trace from previous location --- at NetEscapades.AspNetCore.SecurityHeaders.SecurityHeadersMiddleware.Invoke(HttpContext context) in /_/src/NetEscapades.AspNetCore.SecurityHeaders/SecurityHeadersMiddleware.cs:line 68 at Site.Core.Middleware.WebPMiddleware.Invoke(HttpContext httpContext) in C:\home\site\repository\src\Site.Core\Middleware\WebpMiddleware.cs:line 34 at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddlewareImpl.<Invoke>g__Awaited|10_0(ExceptionHandlerMiddlewareImpl middleware, HttpContext context, Task task) `

Specifics

No response

Steps to reproduce

N/A

Expected result / actual result

No response

madanl1232 avatar Sep 15 '25 14:09 madanl1232

Hi there @madanl1232!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

  • We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
  • If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
  • We'll replicate the issue to ensure that the problem is as described.
  • We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face:

github-actions[bot] avatar Sep 15 '25 14:09 github-actions[bot]

Hi there, we experienced the same on a project running 13.9.3

raouldevries86 avatar Sep 15 '25 15:09 raouldevries86

@madanl1232 @raouldevries86

Do you have any packages installed ? We are not seeing this on our v13 sites.

Dave

dawoe avatar Sep 15 '25 15:09 dawoe

@madanl1232 @raouldevries86

Do you have any packages installed ? We are not seeing this on our v13 sites.

Dave Thx for your response.

It's a pretty basic site. It is a Baseline child project. Packages (most of them are not actively used but are part of our baseline setup):

Umbraco Deploy Umbraco.Deploy.Cloud Version: 13.3.0

Umbraco Forms Umbraco.Forms Version: 13.3.1

Contentment Umbraco.Community.Contentment Version: 5.0.2

UmbracoId Version: 13.1.1

EvolveDashboard

EvolveEditors

HexcoreTemplateLabels Version: 1.0.0

HexcoreUI Version: 1.0.0

TagList

TagManager

TwoFactorProviders

uSync Snapshots Version: 13.1.3

uSync Complete Version: 13.1.3.0

uSyncQueue Version: 13.1.3.0

uSyncRestore Version: 13.1.3.0

uSync Publisher Version: 13.1.3

uSync PeopleEdition Version: 13.1.3

uSync History uSyncHistory Version: 1.0

uSync Exporter Version: 13.1.3

uSync uSync Version: 13.1.3

Iconic

Limbo Tables Limbo.Umbraco.Tables Version: 13.0.0-beta001

Greetings, Raoul

raouldevries86 avatar Sep 15 '25 15:09 raouldevries86

Below is the list of installed packages.

Umbraco Deploy Umbraco.Deploy.Cloud Version: 13.4.0 Workflow Version: 13.3.0 Umbraco Licenses Umbraco.Licenses Version: 13.3.0 Umbraco Forms Umbraco.Forms Version: 13.5.0 Contentment Umbraco.Community.Contentment Version: 4.7.0 UmbracoId Version: 13.2.4 Umbraco.Engage staticwebassets Our.Umbraco.TheDashboard Umbraco Workflow Version: 13.3.0 BlockListFilters Version: 1.0.0 ConditionalDisplayers Diplo.AuditLogViewer Version: 10.1.0 MemberExport TPXimpact.SchemaViewer Version: 1.0.0 addons contentApps dashboard fonts lib styles utils tabFocus Umbraco.Engage.Forms Umbraco.Cms.Integrations.Search.Algolia Version: 1.3.1 UmbracoFormsCustomEntries Umbraco Commerce Deploy Umbraco.Commerce.Deploy Version: 13.1.3 Umbraco Commerce Umbraco.Commerce Version: 13.1.14 AssetBank Media Picker Version: 0.0.2 Skybrud Redirects Import Skybrud.Umbraco.Redirects.Import Version: 13.0.0 Skybrud Redirects Skybrud.Umbraco.Redirects Version: 13.0.5 Our Umbraco TagHelpers Version: 1.2.0.0 Our.Umbraco.MaintenanceMode Dawoe.OEmbedPickerPropertyEditor Version: 13.0.0.0 CMSImport Version: 13.1.4 Mass Participation Dashboard Version: 0.0.1 Hangfire

madanl1232 avatar Sep 16 '25 09:09 madanl1232

We also experience this regularly on multiple websites with Starter and Standard plans. All websites are well within the allowed resources. We don’t use any external packages. All websites were started as baseline clones, but the links have been removed.

• Umbraco.Cloud.Identity.Cms 13.2.4 • Umbraco.Cloud.StorageProviders.AzureBlob 13.1.0 • Umbraco.Cms 13.10.0 • Umbraco.Deploy.Cloud 13.4.0 • Umbraco.Deploy.Contrib 13.3.1 • Umbraco.Forms 13.6.0 • Umbraco.Forms.Core 13.6.0 • Umbraco.Forms.Core.Providers 13.6.0 • Umbraco.Forms.Deploy 13.1.1

orangejuice-martin avatar Sep 23 '25 07:09 orangejuice-martin

We are starting to see this as well on our V13 websites hosted on Azure. So it is not specific to Umbraco Cloud.

I see in this issue more people are experiencing this : https://github.com/dotnet/razor/issues/12062

What version of the .NET are you running ?

Our websites are running 8.0.16

dawoe avatar Sep 25 '25 12:09 dawoe

I asked CoPilot for you. This is what is came back with. Hopefully some of this is helpful to you

Thanks for the detailed context, Paul — that’s really helpful. Based on the error log and your description, here’s a breakdown of what might have caused the issue and why it resolved itself:


🔍 Root Cause Analysis: RegexMatchTimeoutException

This exception typically occurs when:

  • A regex pattern is too complex or inefficient (e.g., nested quantifiers, catastrophic backtracking).
  • The input string is unusually large or malformed, causing the regex engine to exceed its timeout.
  • The timeout threshold for regex operations is too low for the given workload.

In your case, the stack trace points to:

Microsoft.AspNetCore.Razor.Language.HtmlConventions.ToHtmlCase(String name)

This suggests the error occurred during Razor view compilation, likely while parsing or transforming tag helper attributes.


🧩 Why Only Production Was Affected

Here are some plausible reasons:

  1. Traffic Spike or Unusual Input:

    • Production might have received a specific request or content (e.g., from a bot, user input, or external service) that triggered the problematic regex.
    • This could explain why it didn’t happen in dev/staging.
  2. Dynamic Content Differences:

    • Even if the code and content are the same, runtime data (e.g., from external APIs, personalization, or cookies) might differ in production.
  3. View Compilation Race Condition:

    • Razor views are compiled on-demand in production (especially with runtime compilation).
    • A race condition or corrupted cache could have caused a malformed view to be compiled, triggering the regex issue.
  4. Temporary Environment Glitch:

    • A transient issue with the underlying infrastructure (e.g., memory pressure, CPU throttling) could have slowed down regex execution enough to hit the timeout.

Why It Resolved Itself

  • View Recompilation: Razor views might have been recompiled after the initial failure, resolving the issue.
  • Cache Expiry: A corrupted or partial cache entry could have expired or been cleared.
  • Traffic Normalization: The problematic input may have stopped arriving.
  • Umbraco Cloud Infrastructure Recovery: If the issue was due to a temporary glitch in the hosting environment, it may have self-healed.

🛠️ Recommendations

  1. Increase Regex Timeout (if feasible):

    • You can set the timeout globally via AppContext.SetSwitch or per regex instance.
    • But this should be done cautiously — increasing timeout can mask performance issues.
  2. Precompile Razor Views:

    • Consider precompiling views during deployment to avoid runtime compilation issues.
  3. Log Input Causing Regex:

    • If possible, wrap regex calls in try-catch and log the input string when a timeout occurs.
  4. Monitor for Recurrence:

    • Set up alerts for RegexMatchTimeoutException so you can catch it early if it happens again.
  5. Review Custom Tag Helpers:

    • If you have custom tag helpers or Razor components, check for any regex usage that could be optimized.

prjseal avatar Sep 25 '25 12:09 prjseal

One of our production environments, v 13.11.0 hosted on Umbraco Cloud, was also affected by this issue. Restarting the environment resolved the problem temporarily. Do you plan to deliver a long-term fix, and what is the expected timeline? Thanks.

giordanop avatar Nov 19 '25 09:11 giordanop

Looking at the stack trace you're using Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation, which means you still have the default BackofficeDevelopment runtime mode configured. Although this makes it easier to get started and develop Umbraco sites, it also comes with a performance penalty, which is why we recommend production sites to use the Production runtime mode instead, see: https://docs.umbraco.com/umbraco-cms/13.latest/fundamentals/setup/server-setup/runtime-modes. Umbraco 17 even moved support of BackofficeDevelopment runtime mode (and InMemoryAuto Models Builder mode) into a separate package, see the following announcement for more details: https://github.com/umbraco/Announcements/issues/28.

So the first thing everyone encountering this issue should try is to configure Development/Production runtime mode per linked documentation. If you still encounter this exception, you're most likely running the site on underpowered resources (the RegexMatchTimeoutException is very similar to the OutOfMemoryException in that regard). Other improvements could be made in the views themselves, e.g. using async partial/view component rendering, etc.

ronaldbarendse avatar Dec 04 '25 10:12 ronaldbarendse