cli All CLI commands are excruciatingly slow when running on a large repo

Describe the bug

I have a repo with 50K+ files in it. Running any netlify command in that directory (e.g. netlify status) takes about 4 minutes to start, after which the command terminates quickly. Rerunning the command takes 5 minutes again. The systme has 1 CPU maxed out during this time.

E.g.

$ time netlify status
──────────────────────┐
 Current Netlify User │
──────────────────────┘
[...]
────────────────────┐
 Netlify Site Info  │
────────────────────┘
[...]

real	4m5.076s
user	4m6.618s
sys	0m0.931s

Steps to reproduce

Create a huge repo with 50K files, run netlify link and then netlify status.

Configuration

Defaults, didn't configure anything

Environment

System: OS: Linux 5.4 Ubuntu 20.04.4 LTS (Focal Fossa) CPU: (8) x64 Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz Memory: 3.08 GB / 11.16 GB Container: Yes Shell: 5.0.17 - /bin/bash

Jun 27 '22 12:06 karalabe

E.g. Just running netlify lm which just prints the help screen:

$ time netlify lm
Handle Netlify Large Media operations

USAGE
  $ netlify lm [options]

OPTIONS
  -h, --help                                display help for command
  --debug                                   Print debugging information
  --httpProxy [address]                     Proxy server address to route requests through.
  --httpProxyCertificateFilename [file]     Certificate file to use when connecting using a proxy server

DESCRIPTION
  The lm command will help you manage large media for a site

EXAMPLES
  $ netlify lm:info
  $ netlify lm:install
  $ netlify lm:setup

COMMANDS
  $ lm:info                                 Show large media requirements information.
  $ lm:install                              Configures your computer to use Netlify Large Media
  $ lm:setup                                Configures your site to use Netlify Large Media


real	3m53.309s
user	3m54.685s
sys	0m1.563s

I.e. 4 minute hang.

If I run the same command from a different folder it takes 3 seconds for it to run.

Jun 27 '22 12:06 karalabe

Thank you for reporting this. We are aware of some smaller performance issues (#349) but not to that extend. I tried to reproduce this, without success so far.

Is the repository you are talking about public or can be shared?
Are the 50k files source files of some kind (js, ts, go, ...) or static files (HTML, css) which get deployed directly?
Would you be able to create and share a CPU profile with node --prof netlify status? This creates a file isolate-0xxxxxxxx-xxxxxxx-v8.log or similar in the current working directory.
Which version of nodejs are you using? node --version
Do you have some custom settings in the config file netlify.toml?

Jul 28 '22 11:07 danez

Experience for me, while not as horrific as mentioned above, it still slow, given I have a single directory with a couple of HTML files, CSS, favicon, _headers, and _redirects, plus netlify.toml and .netlify directory.

% time ntl
# Removed output
ntl  1.29s user 0.39s system 37% cpu 4.439 total

Subsequent calls a much quicker

% time ntl
# Removed output
ntl  1.25s user 0.27s system 120% cpu 1.256 total

Compare this to Wrangler2

% time wrangler
# Removed output
wrangler  0.42s user 0.12s system 80% cpu 0.668 total

Would you be able to create and share a CPU profile with node --prof netlify status? This creates a file isolate-0xxxxxxxx-xxxxxxx-v8.log or similar in the current working directory.

Running this command throws an error for me

% node --prof netlify    
node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module '/Users/coel/path/to/project'
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
    at node:internal/main/run_main_module:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Which version of nodejs are you using? node --version

% node --version
v16.16.0

Is the repository you are talking about public or can be shared?

Using this repository as another test case I see

% time ntl
# Removed output
ntl  1.35s user 0.41s system 26% cpu 6.655 total

Running the same command again is considerably quicker

% time ntl
# Removed output
ntl  1.22s user 0.28s system 119% cpu 1.255 total

Sep 30 '22 00:09 coelmay

In devcontainer with pnpm monorepo netlify takes around 16 seconds for me:

$ time pnpm netlify

⬥ Netlify CLI
Read the docs: https://docs.netlify.com/cli/get-started/
Support and bugs: https://github.com/netlify/cli/issues

VERSION
  netlify-cli/12.0.9 linux-arm64 node-v18.10.0

...


real    0m16.842s
user    0m2.336s
sys     0m1.595s

Are the 50k files source files of some kind (js, ts, go, ...) or static files (HTML, css) which get deployed directly?

No

Would you be able to create and share a CPU profile with node --prof netlify status? This creates a file isolate-0xxxxxxxx-xxxxxxx-v8.log or similar in the current working directory.

isolate-0x3c913fb0-94864-v8.log

Which version of nodejs are you using? node --version

Node.js v18.10.0

Do you have some custom settings in the config file netlify.toml?

No

Oct 14 '22 19:10 devidw