Layers make nuxt scan too many directories, if the layer has non app related sibling directories

[fabianwohlfart]

Environment

---

• Operating System: Darwin
• Node Version: v22.11.0
• Nuxt Version: 3.14.1592
• CLI Version: 3.16.0
• Nitro Version: 2.10.4
• Package Manager: [email protected]
• Builder: -
• User Config: default
• Runtime Modules: -
• Build Modules: -

---

Reproduction

https://github.com/fabianwohlfart/test__nuxt-layers-slowdown

Describe the bug

I have a layer repo which is used by multiple websites. It it located as a sibling / parallel to all the other website folders.

If I move this layer repo inside my website layer everything works as expected.

Development
- website-1
- - layer (works)
- website-2
- …

The moment I move it to a parent folder nuxt slows down with the warning

Development
- layer (slow)
- website-1
- website-2
- …

WARN Slow module imports took 10219.28ms to setup.

The moment I move it to my User folder the OS asks me for permission to access certain folders. That's why I assume that siblings are scanned too.

User
- Development
- layer (asks for permissions)
- Dokuments
- Images

As it is not possible for me to give you a reproduction with all my folders. Yet I simplified it to a setup where we have one folder as a sibling with 250k js files. If this folder exists it slows down nuxt significantly, but should not, as I don't expect it to be part of my app.

- app
- layer
- should-not-matter
- - [number].js (x 250000)

Without this folder

✔ Vite client built in 25ms
✔ Vite server built in 241ms
✔ Nuxt Nitro server built in 453 ms
ℹ Vite client warmed up in 0ms
ℹ Vite server warmed up in 308ms

With this folder

✔ Vite client built in 20ms                                                                                                  
✔ Vite server built in 172ms                                                                                             
✔ Nuxt Nitro server built in 1717 ms                                                                                 
ℹ Vite client warmed up in 1ms
ℹ Vite server warmed up in 325ms 

Original speed when my layer repo is a sibling of my website repos

 WARN  Slow module imports took 10219.28ms to setup.

ℹ Re-optimizing dependencies because lockfile has changed
✔ Vite client built in 26ms
✔ Vite server built in 348ms
✔ Nuxt Nitro server built in 20094 ms
ℹ Vite client warmed up in 0ms
ℹ Vite server warmed up in 440ms 

Additional context

No response

Logs

[Aareksio]

Since you pinged the nitro PR, this is the inline rollup plugin pi0 created, which counts number of resolved files during the build phase. If you suspect your issue has something to do with scanning unrelated files, it may help you confirm the theory. Furthermore you can modify it slightly to log the resolved path and look for something which shouldn't be there (usually node_modules):

  rollupConfig: {
    plugins: [
      (() => {
        let count = 0;
        const ids = new Set();
        return {
          name: "resolve-counter",
          resolveId(id) {
            ids.add(id);
            count++;
          },
          generateBundle() {
            console.log(
              `🚀 Total ${count} modules resolved, ${ids.size} unique`
            );
          },
        };
      })(),
    ],
  },

[fabianwohlfart]

Since you pinged the nitro PR, this is the inline rollup plugin pi0 created, which counts number of resolved files during the build phase. If you suspect your issue has something to do with scanning unrelated files, it may help you confirm the theory. Furthermore you can modify it slightly to log the resolved path and look for something which shouldn't be there (usually node_modules):

rollupConfig: { plugins: [ (() => { let count = 0; const ids = new Set(); return { name: "resolve-counter", resolveId(id) { ids.add(id); count++; }, generateBundle() { console.log( 🚀 Total ${count} modules resolved, ${ids.size} unique ); }, }; })(), ], },

Maybe i was a bit too hasty on the ping. I tried your plugin, but it logs modules after i get the slow modules import warning. So i guess it's not the right timing.

 WARN  Slow module imports took 11316.5ms to setup.                                                                                                                                                          

ℹ Building client...                                                                                                                                                                                    
ℹ vite v5.4.11 building for production...                                                                                                                                                                   
ℹ ✓ 419 modules transformed.                                                                                                                                                                               
rendering chunks (27)...
🚀 Total 17 modules resolved, 8 unique

[fabianwohlfart]

I tracked it down to the usage of scanDirExports: https://github.com/nuxt/nuxt/blob/main/packages/nuxt/src/imports/module.ts The issue is resolved if we go back to unimport 3.13.4 – there was a switch to tinyglobby in 3.14.0, i (highly) suspect this as the cause

  "devDependencies": {
    "unimport": "3.13.4"
  },
  "overrides": {
    "unimport": "3.13.4"
  }

[davidstackio]

I'm also experiencing this issue. Downgrading unimport as @fabianwohlfart suggested worked for me.

This was the build time of my layer:

[fabianwohlfart]

I'm also experiencing this issue. Downgrading unimport as @fabianwohlfart suggested worked for me.

This was the build time of my layer:

Dev from tinyglobby got back to me. @davidstackio You can use this, until they release the new version and dependents update.

"overrides": {
  "tinyglobby": "https://pkg.pr.new/tinyglobby@d51e2b9"
}

[davidstackio]

@fabianwohlfart Thanks! For my learning, what is the advantage of using the tinyglobby vs the unimport override?

[fabianwohlfart]

@davidstackio My take is that it's better to use a future version of something instead of locking into an old version.

[danielroe]

cc: @antfu

[antfu]

Linking to the upstream issue: https://github.com/SuperchupuDev/tinyglobby/issues/77

Will update the deps when they release a stable new version

[danielroe]

This is also the reason we cannot upgrade to latest unimport in https://github.com/nuxt/nuxt/pull/30182 - it slowed each fixture from <500ms to >8s (in simply running loadNuxt on the repository root).

[antfu]

[email protected] migrated back to fast-glob to workaround it

[denishsharma]

Downgrading to previous version and updating to latest version of the unimport doesn't work for the layer with nuxt 4 directory structure. I have added link to the test repo, it is created using nuxt.new layer template and uses nuxt v4 directory structure.

denishsharma/nuxt-layer-startup-time-issue-repo

With Nuxt 4 directory structure

The below one is with the workaround provided above.

Without Nuxt 4 directory structure (but uses compatibilityVersion: 4)

[danielroe]

@denishsharma to benefit from the v4 features, make sure your layer is also using compatibilityVersion: 4 and the new directory structure.

[denishsharma]

@danielroe exponential increase in startup time using layers with v4 compatibility still persists, even after updating my layer and project to use compatibilityVersion: 4 and the new directory structure.

Fallback to using v3 directory structure for layer and project, resolves this issue, regardless of setting compatibilityVersion to 4.

Tested both on latest version of nuxt which is 3.15.1 at date of this comment.

[danielroe]

@denishsharma The latest version of Nuxt is v3.15.2 (released two days ago) which contains some speed up but I think the issue here might be resolved in https://github.com/nuxt/nuxt/pull/30620 - I'll need to confirm, or you can try with npm:nuxt-nightly@3x.

[denishsharma]

@danielroe issue still persists even with nightly release npm:nuxt-nighlty@3x. Below are the differences of startup time, first is using old directory structure (with compatibilityVersion:4 <- this is regardless of startup time, only directory structure makes the difference) and second one is using new directory structure. (both uses 3.x nightly release of nuxt) (this test is performed on simple nuxt layer with .playground folder)

Using old directory structure

Using new directory structure

[fabianwohlfart]

I could also check it if it's related to my original problem. yet i would need a reproduction. can you share your code in a github repo or a stackblitz?

[denishsharma]

@danielroe @fabianwohlfart I have encountered strange behaviour with layers (with new directory structure), somehow, when I add empty "app" folder, the exponential increase in startup time resolves. I have no idea if this is a bug, or "app" folder is mandatory with new directory structure.

Here is the demo repo: (playground does not even use anything from layer, nor does layer exports anything) denishsharma/nuxt-layer-startup-time-issue-repo

When there is no app folder inside layer

When there is app folder insider layer, this app folder is empty

(i have added .gitkeep so that i can push the app folder)

[benmccann]

tinyglobby 0.2.11 is out, which fixes all known performance issues there. I sent https://github.com/unjs/unimport/pull/431 to switch unimport back to tinyglobby. Are you able to test it out against nuxt to ensure it won't cause any issues?

[fabianwohlfart]

Yes, works

tinyglobby 0.2.10 unimport 3.14.4

 WARN  Slow module imports took 13558.62ms to setup.  

✔ Vite client built in 66ms
✔ Vite server built in 327ms
✔ Nuxt Nitro server built in 27604 ms
ℹ Vite client warmed up in 1ms
ℹ Vite server warmed up in 467ms

tinyglobby 0.2.11 unimport 3.14.4

✔ Vite client built in 44ms
✔ Vite server built in 389ms
✔ Nuxt Nitro server built in 566 ms
ℹ Vite client warmed up in 0ms 
ℹ Vite server warmed up in 496ms