cli icon indicating copy to clipboard operation
cli copied to clipboard

--watch will error if the folder contains too many files

Open JakeChampion opened this issue 2 years ago • 4 comments

My project has lots of files (214,269 files) and when I try and use the --watch mode, the cli will error

Version

❯ fastly version Fastly CLI version v10.4.0 (4b869ab3) Built with go version go1.21.0 linux/amd64 Viceroy version: viceroy 0.7.0

What happened

I ran fastly compute serve --watch and had this error happen:

2023/09/06 02:20:55 error configuring watching for file changes: open polyfill-libraries/polyfill-library-3.52.1/polyfills/__dist/Intl.PluralRules.~locale.os: too many open files

JakeChampion avatar Sep 06 '23 01:09 JakeChampion

@JakeChampion

214,269 files

Why Jake, why!? 😂

Integralist avatar Sep 06 '23 08:09 Integralist

Nearly all of those files are used for KV Store entries ☺️

JakeChampion avatar Sep 06 '23 09:09 JakeChampion

We typically recommend defining a .fastlyignore and adding patterns for files/directories to skip. I appreciate that's not going to necessarily help your particular issue though.

Integralist avatar Sep 06 '23 09:09 Integralist

Have you tried bumping your file descriptor limits (ulimit -n)? Even if this works I suspect keeping tabs of so many files is going to be very slow.

fgsch avatar Sep 22 '23 14:09 fgsch