esbuild icon indicating copy to clipboard operation
esbuild copied to clipboard

[Bug?] Esbuild slow performance due to discarding dir cache when the plugin calls the built-in resolver

Open pyrocat101 opened this issue 8 months ago • 0 comments

Description

If the resolve plugin fallbacks to use the built-in resolver, a new resolver is created, causing dir cache to be rebuilt: https://github.com/evanw/esbuild/blob/67cbf87a4909d87a902ca8c3b69ab5330defab0a/pkg/api/api_impl.go#L2029-L2031

When there are a large number of files to be resolved, rebuilding these dir cache takes significant numbers of time, as is demonstrated in the repro below.

Repro

https://github.com/pyrocat101/esbuild-repro

This repro uses esbuild to bundle lodash-es, which has a couple hundred files to demonstrate the performance degration. The resolve plugin simply forwards the resolve request to the built-in esbuild resolver. When this plugin is enabled, it takes ~2.5s to bundle on my machine. But when the plugin is disabled, it takes ~0.2s.

I collected pprof cpu profile from esbuild, and it shows that most of time is spent in dirInfoUncached due to having to repopulate the resolver dir cache on every resolve call.

Impact

This issue affects use cases such as https://github.com/aspect-build/rules_esbuild, which uses a resolve plugin to make sure esbuild does not chase symlink out of the sandbox by post-processing the built-in resolver's result.

Potential Fix

Hoist the resolver.NewResolver call out of the resolve function definition to avoid creating it on every call. I built esbuild with the following patch applied and can confirm the performance has significantly improved:

diff --git a/pkg/api/api_impl.go b/pkg/api/api_impl.go
index 843f7f81..607d60cb 100644
--- a/pkg/api/api_impl.go
+++ b/pkg/api/api_impl.go
@@ -1994,11 +1994,17 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
 
 	var optionsForResolve *config.Options
 	var plugins []config.Plugin
+	var res *resolver.Resolver
 
 	// This is called after the build options have been validated
 	finalizeBuildOptions = func(options *config.Options) {
 		options.Plugins = plugins
 		optionsForResolve = options
+
+		// Make a new resolver so it has its own log
+		log := logger.NewDeferLog(logger.DeferLogNoVerboseOrDebug, validateLogOverrides(initialOptions.LogOverride))
+		optionsClone := *optionsForResolve
+		res = resolver.NewResolver(config.BuildCall, fs, log, caches, &optionsClone)
 	}
 
 	for i, item := range clone {
@@ -2025,11 +2031,6 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
 				return ResolveResult{Errors: []Message{{Text: "Must specify \"kind\" when calling \"resolve\""}}}
 			}
 
-			// Make a new resolver so it has its own log
-			log := logger.NewDeferLog(logger.DeferLogNoVerboseOrDebug, validateLogOverrides(initialOptions.LogOverride))
-			optionsClone := *optionsForResolve
-			resolver := resolver.NewResolver(config.BuildCall, fs, log, caches, &optionsClone)
-
 			// Make sure the resolve directory is an absolute path, which can fail
 			absResolveDir := validatePath(log, fs, options.ResolveDir, "resolve directory")
 			if log.HasErrors() {
@@ -2043,7 +2044,7 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
 			kind := resolveKindToImportKind(options.Kind)
 			resolveResult, _, _ := bundler.RunOnResolvePlugins(
 				plugins,
-				resolver,
+				res,
 				log,
 				fs,
 				&caches.FSCache,
@@ -2074,7 +2075,7 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
 				if options.PluginName != "" {
 					pluginName = options.PluginName
 				}
-				text, _, notes := bundler.ResolveFailureErrorTextSuggestionNotes(resolver, path, kind, pluginName, fs, absResolveDir, optionsForResolve.Platform, "", "")
+				text, _, notes := bundler.ResolveFailureErrorTextSuggestionNotes(res, path, kind, pluginName, fs, absResolveDir, optionsForResolve.Platform, "", "")
 				result.Errors = append(result.Errors, convertMessagesToPublic(logger.Error, []logger.Msg{{
 					Data:  logger.MsgData{Text: text},
 					Notes: notes,

pyrocat101 avatar Jun 06 '24 19:06 pyrocat101