sbt-native-packager
sbt-native-packager copied to clipboard
sbt
“Too many open files in system” exception when mapping a directory
So when I'm trying to map a couple additional directories to be included in the package, then the packaging fails with this:
Link to the question on StackOverflow: http://stackoverflow.com/q/37875451/1296362
Could not create file [...omitted...]/target/streams/compile/$global/$global/discoveredMainClasses/datajava.io.IOException: Too many open files in system
at sbt.ErrorHandling$.translate(ErrorHandling.scala:10)
at sbt.IO$.touch(IO.scala:196)
at sbt.std.Streams$$anon$3$$anon$2.make(Streams.scala:129)
at sbt.std.Streams$$anon$3$$anon$2.binary(Streams.scala:116)
at sbt.SessionVar$$anonfun$persist$1.apply(SessionVar.scala:27)
at sbt.SessionVar$$anonfun$persist$1.apply(SessionVar.scala:26)
at sbt.std.Streams$class.use(Streams.scala:75)
at sbt.std.Streams$$anon$3.use(Streams.scala:100)
at sbt.SessionVar$.persist(SessionVar.scala:26)
at sbt.SessionVar$.persistAndSet(SessionVar.scala:21)
at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$1$$anonfun$apply$5.apply(Project.scala:563)
at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$1$$anonfun$apply$5.apply(Project.scala:563)
at sbt.SessionVar$$anonfun$1$$anonfun$apply$1.apply(SessionVar.scala:40)
at sbt.SessionVar$$anonfun$1$$anonfun$apply$1.apply(SessionVar.scala:40)
at scala.Function$$anonfun$chain$1$$anonfun$apply$1.apply(Function.scala:24)
at scala.Function$$anonfun$chain$1$$anonfun$apply$1.apply(Function.scala:24)
at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:51)
at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:60)
at scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.$div$colon(TraversableOnce.scala:138)
at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:105)
at scala.Function$$anonfun$chain$1.apply(Function.scala:24)
at sbt.EvaluateTask$.applyResults(EvaluateTask.scala:387)
at sbt.EvaluateTask$.liftedTree1$1(EvaluateTask.scala:361)
at sbt.EvaluateTask$.run$1(EvaluateTask.scala:358)
at sbt.EvaluateTask$.runTask(EvaluateTask.scala:378)
at sbt.Aggregation$$anonfun$3.apply(Aggregation.scala:64)
at sbt.Aggregation$$anonfun$3.apply(Aggregation.scala:62)
at sbt.EvaluateTask$.withStreams(EvaluateTask.scala:314)
at sbt.Aggregation$.timedRun(Aggregation.scala:62)
at sbt.Aggregation$.runTasks(Aggregation.scala:71)
at sbt.Aggregation$$anonfun$applyTasks$1.apply(Aggregation.scala:32)
at sbt.Aggregation$$anonfun$applyTasks$1.apply(Aggregation.scala:31)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Aggregation$$anonfun$evaluatingParser$4$$anonfun$apply$5.apply(Aggregation.scala:153)
at sbt.Aggregation$$anonfun$evaluatingParser$4$$anonfun$apply$5.apply(Aggregation.scala:152)
at sbt.Act$$anonfun$sbt$Act$$actParser0$1$$anonfun$sbt$Act$$anonfun$$evaluate$1$1$$anonfun$apply$10.apply(Act.scala:244)
at sbt.Act$$anonfun$sbt$Act$$actParser0$1$$anonfun$sbt$Act$$anonfun$$evaluate$1$1$$anonfun$apply$10.apply(Act.scala:241)
at sbt.Command$.process(Command.scala:93)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.State$$anon$1.process(State.scala:184)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.MainLoop$.next(MainLoop.scala:96)
at sbt.MainLoop$.run(MainLoop.scala:89)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:68)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:63)
at sbt.Using.apply(Using.scala:24)
at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
at sbt.MainLoop$.runLogged(MainLoop.scala:22)
at sbt.StandardMain$.runManaged(Main.scala:54)
at sbt.xMain.run(Main.scala:29)
at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
at xsbt.boot.Launch$.run(Launch.scala:109)
at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
at xsbt.boot.Launch$.launch(Launch.scala:117)
at xsbt.boot.Launch$.apply(Launch.scala:18)
at xsbt.boot.Boot$.runImpl(Boot.scala:41)
at xsbt.boot.Boot$.main(Boot.scala:17)
at xsbt.boot.Boot.main(Boot.scala)
Caused by: java.io.IOException: Too many open files in system
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1012)
at sbt.IO$$anonfun$1.apply$mcZ$sp(IO.scala:196)
at sbt.IO$$anonfun$1.apply(IO.scala:196)
at sbt.IO$$anonfun$1.apply(IO.scala:196)
at sbt.ErrorHandling$.translate(ErrorHandling.scala:10)
at sbt.IO$.touch(IO.scala:196)
at sbt.std.Streams$$anon$3$$anon$2.make(Streams.scala:129)
at sbt.std.Streams$$anon$3$$anon$2.binary(Streams.scala:116)
at sbt.SessionVar$$anonfun$persist$1.apply(SessionVar.scala:27)
at sbt.SessionVar$$anonfun$persist$1.apply(SessionVar.scala:26)
at sbt.std.Streams$class.use(Streams.scala:75)
at sbt.std.Streams$$anon$3.use(Streams.scala:100)
at sbt.SessionVar$.persist(SessionVar.scala:26)
at sbt.SessionVar$.persistAndSet(SessionVar.scala:21)
at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$1$$anonfun$apply$5.apply(Project.scala:563)
at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$1$$anonfun$apply$5.apply(Project.scala:563)
at sbt.SessionVar$$anonfun$1$$anonfun$apply$1.apply(SessionVar.scala:40)
at sbt.SessionVar$$anonfun$1$$anonfun$apply$1.apply(SessionVar.scala:40)
at scala.Function$$anonfun$chain$1$$anonfun$apply$1.apply(Function.scala:24)
at scala.Function$$anonfun$chain$1$$anonfun$apply$1.apply(Function.scala:24)
at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:51)
at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:60)
at scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.$div$colon(TraversableOnce.scala:138)
at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:105)
at scala.Function$$anonfun$chain$1.apply(Function.scala:24)
at sbt.EvaluateTask$.applyResults(EvaluateTask.scala:387)
at sbt.EvaluateTask$.liftedTree1$1(EvaluateTask.scala:361)
at sbt.EvaluateTask$.run$1(EvaluateTask.scala:358)
at sbt.EvaluateTask$.runTask(EvaluateTask.scala:378)
at sbt.Aggregation$$anonfun$3.apply(Aggregation.scala:64)
at sbt.Aggregation$$anonfun$3.apply(Aggregation.scala:62)
at sbt.EvaluateTask$.withStreams(EvaluateTask.scala:314)
at sbt.Aggregation$.timedRun(Aggregation.scala:62)
at sbt.Aggregation$.runTasks(Aggregation.scala:71)
at sbt.Aggregation$$anonfun$applyTasks$1.apply(Aggregation.scala:32)
at sbt.Aggregation$$anonfun$applyTasks$1.apply(Aggregation.scala:31)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Aggregation$$anonfun$evaluatingParser$4$$anonfun$apply$5.apply(Aggregation.scala:153)
at sbt.Aggregation$$anonfun$evaluatingParser$4$$anonfun$apply$5.apply(Aggregation.scala:152)
at sbt.Act$$anonfun$sbt$Act$$actParser0$1$$anonfun$sbt$Act$$anonfun$$evaluate$1$1$$anonfun$apply$10.apply(Act.scala:244)
at sbt.Act$$anonfun$sbt$Act$$actParser0$1$$anonfun$sbt$Act$$anonfun$$evaluate$1$1$$anonfun$apply$10.apply(Act.scala:241)
at sbt.Command$.process(Command.scala:93)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.State$$anon$1.process(State.scala:184)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.MainLoop$.next(MainLoop.scala:96)
at sbt.MainLoop$.run(MainLoop.scala:89)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:68)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:63)
at sbt.Using.apply(Using.scala:24)
at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
at sbt.MainLoop$.runLogged(MainLoop.scala:22)
at sbt.StandardMain$.runManaged(Main.scala:54)
at sbt.xMain.run(Main.scala:29)
at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
at xsbt.boot.Launch$.run(Launch.scala:109)
at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
at xsbt.boot.Launch$.launch(Launch.scala:117)
at xsbt.boot.Launch$.apply(Launch.scala:18)
at xsbt.boot.Boot$.runImpl(Boot.scala:41)
at xsbt.boot.Boot$.main(Boot.scala:17)
at xsbt.boot.Boot.main(Boot.scala)
Thanks for your report. I'm not sure if this is a native-packager related issue. The stacktrace doesn't seem to include a hint to native packager. I haven't use the dist command myself. Can you try the following commands and see if the error persists:
universal:packageBinstage
I don't quite see the difference between dist and universal:packageBin commands. Here is the output of the second one:
[info] Wrote [...omitted...]/modules/cloud-service/target/scala-2.11/cloud-service_2.11-0.1-SNAPSHOT.pom
[info] Wrote [...omitted...]/modules/payments-service/target/scala-2.11/payments-service_2.11-0.1-SNAPSHOT.pom
[info] Wrote [...omitted...]/modules/newsFeed-manager/target/scala-2.11/news-feed_2.11-0.1-SNAPSHOT.pom
Could not create file [...omitted...]/modules/company-manager/target/streams/compile/$global/$global/discoveredMainClasses/datajava.io.IOEx
ception: Too many open files in system
So the problem is in number of open files I think, because sometimes (rarely) it works.
Partially I fixed the problem by raising hard and soft limit on the Linux system with ulimit utility. But on local OS X machine there is no such limit at all though such failures still occur.
@muuki88 I've seen this sometimes, but in my case it was along with likely badly behaving plugin that I wrote. The stack trace is failing to create /target/streams/compile/$global/$global/discoveredMainClasses/data, which is something used internally by sbt but that doesn't tell much apart from the fact that there must have been many files already opened before it hit the error.
@ilyai Do you have something we can use to reproduce this issue on our machines to study this further? Are you running this on a single subprojects?
@eed3si9n Here are the links to build.sbt and plugins.sbt of the project. To reproduce try to add to mappings a few directories with lots of files (mappings in Universal ++= directory("somedir1")). In the project I have over 40,000 files, but most of them are front-end related npm modules and I need them to be in package.
I have the same problem while building https://github.com/yahoo/kafka-manager:
Could not create file /home/kosiakk/kafka-manager-1.3.3.15/target/streams/compile/$global/$global/discoveredMainClasses/datajava.io.IOException: Too many open files
On my particularly large server, cat /proc/sys/fs/file-max yields 52307681 - that's 52 million files limit.
Additionally, setting ulimit 999999 didn't help.
In the end, I managed to assemble the project on my Windows machine.
