wails icon indicating copy to clipboard operation
wails copied to clipboard

macOS: FATAL: too many open files

Open iansinnott opened this issue 3 years ago • 12 comments

Description

The build system fails with an error about too many files. The frontend build and wails runtime will continue and you will have to close them manually.

Solution: Find out where you have a ton of files and remove them.

This seems to be an issue with the file watcher erroring out when there are many files within the watch path. In my case the path was frontend/tmp, which was gitignored but being watched by the wails fs watcher.

To Reproduce

Put a bunch of files in your frontend directory.

Expected behaviour

Ideally:

  • everything fails at once (i.e. vite process does not continue in the background)
  • the error message is more descriptive

Might also be useful to support ignore patterns for the file watcher.

Screenshots

No response

Attempted Fixes

No response

System Details

System
------
OS:		MacOS
Version: 	12.5
ID:		21G72
Go Version:	go1.18
Platform:	darwin
Architecture:	arm64

Wails
------
Version: 	v2.0.0-beta.43

Dependency			Package Name	Status		Version
----------			------------	------		-------
xcode command line tools 	N/A		Installed	2395
npm 				N/A		Installed	8.5.0
*upx 				N/A		Available
*nsis 				N/A		Available

Additional context

No response

iansinnott avatar Aug 12 '22 07:08 iansinnott

I only opened this to have a record for anyone else who, like myself, was temporarily mystified by the too many files error.

iansinnott avatar Aug 12 '22 07:08 iansinnott

This might be project specific and related to fsnotify. Investigating 👍

leaanthony avatar Aug 12 '22 15:08 leaanthony

@leaanthony FWIW - MacOS does have an issue with open file limits - is this maybe linked?

This screenshot sourced from the nw-builder project which had the problem in the early days.

ulimit

Blatman avatar Aug 18 '22 14:08 Blatman

Many projects have this issue due to the fact that fsnotify uses a file handle per file. The solution is to use FSEvents on Mac. Need time to do it

leaanthony avatar Aug 18 '22 20:08 leaanthony

Maybe if the watcher would automatically ignore everything in .gitignore it would solve 90% of cases? This would have solved my own issue above.

Or perhaps an -ignore option for wails dev? the -reloaddirs flag seems only only allow watching more files.

A mac-specific solution like FSEvents would be great but I suspect would be more involved.

iansinnott avatar Aug 19 '22 01:08 iansinnott

. Directories should be ignored. If not we should add it. The default open file handle limit on Mac is 256 😬

leaanthony avatar Aug 19 '22 07:08 leaanthony

Re-opening this. I want it fixed properly without workarounds.

leaanthony avatar Aug 29 '22 21:08 leaanthony

@iansinnott - this PR adds support for reading the default .gitignore files and adds them to the paths to be ignored by the watcher. Please let me know how you get on 👍

leaanthony avatar Sep 03 '22 04:09 leaanthony

Cheers! Thank you for working on this issue and thanks for the ongoing work on Wails

iansinnott avatar Sep 03 '22 04:09 iansinnott

@Blatman - does this PR fix your original problem?

leaanthony avatar Sep 08 '22 11:09 leaanthony

Can't say as I haven't encountered it with WAILS yet (only with nw-builder way back) but it looks a good enhancement to avoid a fatal in WAILS.

Blatman avatar Sep 09 '22 15:09 Blatman

@iansinnott - just wondering if you had any feedback on this. Thank you 🙏

leaanthony avatar Sep 11 '22 02:09 leaanthony

Please feel free to reopen this if it's still an issue 👍

leaanthony avatar Sep 22 '22 09:09 leaanthony