gostatic
gostatic copied to clipboard
too many open files
I was trying to build your site but when serving the pages I get
$ make w
gostatic -w config
Rendering 0 changed pages of 292 total
Starting server at *:8000...
2016/07/04 22:37:34 http: Accept error: accept tcp [::]:8000: accept: too many open files; retrying in 5ms
2016/07/04 22:37:34 http: Accept error: accept tcp [::]:8000: accept: too many open files; retrying in 10ms
Building it and running python -m SimpleHTTPServer 8000
instead works fine though.
Running ulimit -S -n 4096
will fix that. gostatic opens all files and it takes time to GC them it seems, so it can blow with such an error.
OK, let's rephrase that then: why does it open all files and keep them open?
I think first part is obvious, but second one is not. :) I did look at why it opens them IIRC, but I didn't action on it. Not sure why it keeps them open though, that needs to be researched again it seems. :)
I think first part is obvious, but second one is not. :)
true :)
that needs to be researched again it seems.
care to re-open then?
Sure! :)) If you want to research/fix that, I would really appreciate it! If not, I guess I'll look at that sometime, but not sure when - a bit too busy lately with work and real-life stuff.
Had a look at the FAQ for fsnotify (which is imported by lib/watch.go)
How many files can be watched at once?
There are OS-specific limits as to how many watches can be created:
Linux: /proc/sys/fs/inotify/max_user_watches contains the limit, reaching this limit results in a "no space left on device" error. BSD / OSX: sysctl variables "kern.maxfiles" and "kern.maxfilesperproc", reaching these limits results in a "too many open files" error.
As long as you're watching them, the files have to be open. It's easiest to increase the limit.
Alternatively you could use a "polling" approach which is way slower, but will work for any amount of files.
Well, that statement isn't true for all OS. On macOS it would be great to use FSEvents
.
https://github.com/emcrisostomo/fswatch
But it sounds like there is a progress in supporting that with fsnotify
.
https://github.com/golang/go/issues/4068
...and specifically https://github.com/fsnotify/fsnotify/issues/11