esbuild
esbuild copied to clipboard
[Bug?] Esbuild slow performance due to discarding dir cache when the plugin calls the built-in resolver
Description
If the resolve plugin fallbacks to use the built-in resolver, a new resolver is created, causing dir cache to be rebuilt: https://github.com/evanw/esbuild/blob/67cbf87a4909d87a902ca8c3b69ab5330defab0a/pkg/api/api_impl.go#L2029-L2031
When there are a large number of files to be resolved, rebuilding these dir cache takes significant numbers of time, as is demonstrated in the repro below.
Repro
https://github.com/pyrocat101/esbuild-repro
This repro uses esbuild to bundle lodash-es, which has a couple hundred files to demonstrate the performance degration. The resolve plugin simply forwards the resolve request to the built-in esbuild resolver. When this plugin is enabled, it takes ~2.5s to bundle on my machine. But when the plugin is disabled, it takes ~0.2s.
I collected pprof cpu profile from esbuild, and it shows that most of time is spent in dirInfoUncached
due to having to repopulate the resolver dir cache on every resolve call.
Impact
This issue affects use cases such as https://github.com/aspect-build/rules_esbuild, which uses a resolve plugin to make sure esbuild does not chase symlink out of the sandbox by post-processing the built-in resolver's result.
Potential Fix
Hoist the resolver.NewResolver
call out of the resolve
function definition to avoid creating it on every call. I built esbuild with the following patch applied and can confirm the performance has significantly improved:
diff --git a/pkg/api/api_impl.go b/pkg/api/api_impl.go
index 843f7f81..607d60cb 100644
--- a/pkg/api/api_impl.go
+++ b/pkg/api/api_impl.go
@@ -1994,11 +1994,17 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
var optionsForResolve *config.Options
var plugins []config.Plugin
+ var res *resolver.Resolver
// This is called after the build options have been validated
finalizeBuildOptions = func(options *config.Options) {
options.Plugins = plugins
optionsForResolve = options
+
+ // Make a new resolver so it has its own log
+ log := logger.NewDeferLog(logger.DeferLogNoVerboseOrDebug, validateLogOverrides(initialOptions.LogOverride))
+ optionsClone := *optionsForResolve
+ res = resolver.NewResolver(config.BuildCall, fs, log, caches, &optionsClone)
}
for i, item := range clone {
@@ -2025,11 +2031,6 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
return ResolveResult{Errors: []Message{{Text: "Must specify \"kind\" when calling \"resolve\""}}}
}
- // Make a new resolver so it has its own log
- log := logger.NewDeferLog(logger.DeferLogNoVerboseOrDebug, validateLogOverrides(initialOptions.LogOverride))
- optionsClone := *optionsForResolve
- resolver := resolver.NewResolver(config.BuildCall, fs, log, caches, &optionsClone)
-
// Make sure the resolve directory is an absolute path, which can fail
absResolveDir := validatePath(log, fs, options.ResolveDir, "resolve directory")
if log.HasErrors() {
@@ -2043,7 +2044,7 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
kind := resolveKindToImportKind(options.Kind)
resolveResult, _, _ := bundler.RunOnResolvePlugins(
plugins,
- resolver,
+ res,
log,
fs,
&caches.FSCache,
@@ -2074,7 +2075,7 @@ func loadPlugins(initialOptions *BuildOptions, fs fs.FS, log logger.Log, caches
if options.PluginName != "" {
pluginName = options.PluginName
}
- text, _, notes := bundler.ResolveFailureErrorTextSuggestionNotes(resolver, path, kind, pluginName, fs, absResolveDir, optionsForResolve.Platform, "", "")
+ text, _, notes := bundler.ResolveFailureErrorTextSuggestionNotes(res, path, kind, pluginName, fs, absResolveDir, optionsForResolve.Platform, "", "")
result.Errors = append(result.Errors, convertMessagesToPublic(logger.Error, []logger.Msg{{
Data: logger.MsgData{Text: text},
Notes: notes,