erlide_eclipse
erlide_eclipse copied to clipboard
erlang
Eclipse hangs when importing/building very large erlang project
We have a very large erlang project, in the following structure: -- top -----folderA ----------app1 ---------------src, ebin, include, test, ... ----------app2 ---------------src, ebin, include, test, ... ----------appX ---------------src, ebin, include, test, ... -----folderB ----------app1 ---------------src, ebin, include, test, ... ----------app2 ---------------src, ebin, include, test, ... ----------appX ---------------src, ebin, include, test, ... -----folderX ----------app1 ---------------src, ebin, include, test, ... ----------app2 ---------------src, ebin, include, test, ... ----------appX ---------------src, ebin, include, test, ...
and so on. According to the "discover Path" feature I have 119 include folders and 247 src folders. Eclipse never manage to build my project, it hangs forever.
Any help is appreciated.
/Alex
For such a large project, it takes a while to synchronize all resource data and then build.
Do you have "automatic build" enabled? If yes, turn it off and import the project again. Wait until the workspace is quiet and turn the build on again. See what happens - if there is trouble, please mail me the erlide.log file from the workspace directory.
I would say that such a big code base is better suited for multiple smaller projects, but I'm not entirely sure how well Eclipse can handle 247 projects...
Thanks for the tip will try it and report back!
Regarding the 247 projects: Not sure it would be suited to manually import all of them either... :/ I tried using the "working set" feature in eclipse, but it didn't seem like erlide was supporting that? Maybe new issue for this?
If your applications have a standard OTP structure, it's not difficult to write a script that will create the necessary .project file in each folder (they all look the same, only the project name changes). This is all that is needed, because the OTP configuration is the default one.
Please do create a ticket for "working sets", it would be useful to have them.
I disabled the "automatic build" option. Waited until workspace got quite, then associated the project with ClearCase (forgot to mention earlier that source code is under version control by Rational ClearCase, which might delay file access). Then started a manual build, which worked fine (took some time though).
So far so good :)
I have now enabled the "auto build" again. Now it takes about 30 seconds to compile a file each time I press Ctrl-s. Is there anyway to speed up the compile time?
Regarding generating .project files for each application: How to open up the projects in eclipse? Or must I open them one by one in eclipse manually?
It shouldn't take that long to compile a file. I would try with clearcase disabled, it can do weird things. I can look at a log to see if there is anything in it (workspace/.metadata/.log and workspace/erlide.log).
If you generate .project files, then you can use Import->Existing projects and select the parent directory. The wizard will scan it and you get a list of all projects to choose from. Having several projects has also the advantage that you can close most of them if you aren't working with that part of the code, which should speed up things. A disadvantage right now is that if you want to execute your code from erlide, it will be a little more complicated (some manual configuration and all projects need to be open).
I'm not sure I can disable clearcase? I can not find a any .log or erlide.log files in locations mentioned above.
I have now removed my .eclipse folder and started from scratch and see the following exception in the console window where i start Eclipse, is that anything to worry about?
12:43:57,307 S: (ExtensionUtils.java:41) : Plug-in "org.erlide.model" was unable to instantiate class "org.erlide.engine.internal.DefaultErlangEngine".
org.eclipse.core.runtime.CoreException: Plug-in "org.erlide.model" was unable to instantiate class "org.erlide.engine.internal.DefaultErlangEngine".
at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.throwException(RegistryStrategyOSGI.java:194)
at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:186)
at org.eclipse.core.internal.registry.ExtensionRegistry.createExecutableExtension(ExtensionRegistry.java:905)
at org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:243)
at org.eclipse.core.internal.registry.ConfigurationElementHandle.createExecutableExtension(ConfigurationElementHandle.java:55)
at org.erlide.util.services.ExtensionUtils.getSingletonExtension(ExtensionUtils.java:36)
at org.erlide.engine.ModelActivator.start(ModelActivator.java:15)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:711)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:702)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:683)
at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:381)
at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:300)
at org.eclipse.osgi.framework.util.SecureAction.start(SecureAction.java:478)
at org.eclipse.osgi.internal.loader.BundleLoader.setLazyTrigger(BundleLoader.java:263)
at org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter.postFindLocalClass(EclipseLazyStarter.java:109)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:469)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:395)
at org.eclipse.osgi.internal.loader.SingleSourcePackage.loadClass(SingleSourcePackage.java:35)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:461)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.erlide.ui.util.ProblemsLabelDecorator.computeAdornmentFlags(ProblemsLabelDecorator.java:91)
at org.erlide.ui.navigator.NavigatorProblemsDecorator.computeAdornmentFlags(NavigatorProblemsDecorator.java:31)
at org.erlide.ui.util.ProblemsLabelDecorator.decorate(ProblemsLabelDecorator.java:196)
at org.eclipse.ui.internal.decorators.LightweightDecoratorDefinition.decorate(LightweightDecoratorDefinition.java:263)
at org.eclipse.ui.internal.decorators.LightweightDecoratorManager$LightweightRunnable.run(LightweightDecoratorManager.java:81)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at org.eclipse.ui.internal.decorators.LightweightDecoratorManager.decorate(LightweightDecoratorManager.java:365)
at org.eclipse.ui.internal.decorators.LightweightDecoratorManager.getDecorations(LightweightDecoratorManager.java:347)
at org.eclipse.ui.internal.decorators.DecorationScheduler$1.ensureResultCached(DecorationScheduler.java:371)
at org.eclipse.ui.internal.decorators.DecorationScheduler$1.run(DecorationScheduler.java:331)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Caused by: java.lang.NullPointerException
at org.erlide.runtime.runtimeinfo.RuntimeInfo.
Clearcase: depending on which eclipse plugin you use, there might be a button in the toolbar or a menu that allows you to switch it on and off. If there isn't one, you can leave the project not connected to clearcase.
logs: do you look in your workspace dir? it might be called something else then "workspace". There is also a dialog under window->preferences->erlang->report problem where you can put together a report that includes the relevant logs.
runtime: :-)
I removed my workspace completely and created a new one (and re-did the build scenario) and now I have a erlide.log file. However, it is 54MB large! I compressed it to 700 KB. How can I attach files here?
It takes about 20 seconds to compile one file.
/Alex
If the log is that big, you could open it and first look what kind of exceptions do happen and fill the log.
Please select one of the stacktraces and paste it here.
I have 6 of these exceptions in the log. No other exceptions. org.erlide.runtime.rpc.RpcException: {badrpc,{'EXIT',{{badmatch,{error,{10,erl_parse,["syntax error before: ",["_app"]]}}},[{erlide_builder,compile_app_src,3,[]},{rpc,'-handle_call_call/6-fun-0-',5,[{file,"rpc.erl"},{line,205}]}]}}} at org.erlide.runtime.internal.rpc.RpcSite.call(RpcSite.java:167) at org.erlide.runtime.internal.rpc.RpcSite.call(RpcSite.java:182) at org.erlide.runtime.internal.rpc.RpcSite.call(RpcSite.java:261) at org.erlide.core.builder.ErlideBuilder.fillAppFileDetails(ErlideBuilder.java:312) at org.erlide.core.builder.ErlideBuilder.handleAppFile(ErlideBuilder.java:297) at org.erlide.core.builder.ErlideBuilder.build(ErlideBuilder.java:157) at org.erlide.core.internal.builder.ErlangBuilder.build(ErlangBuilder.java:28) at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:733) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:206) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:246) at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:299) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:302) at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:358) at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:381) at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:143) at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:241) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
I have "auto build" enabled. I make a change in a file (add an compilation error(add space in function name)), press Ctrl-S and the following appears in log (cleared the old log first):
11:02:21,642 F: (ErlideBuilder.java:136) : ###** Starting build auto of INT-GTE 11:02:21,652 F: (ErlReconciler.java:719) : reset 11:02:21,652 F: (ErlReconcilingStrategy.java:61) : ## initial reconcile cs.erl 11:02:21,696 F: (LogEventHandler.java:27) : info: {"/vobs/erbs/int/gte/fw/cpp/test/eunit",{ok,["cpp_fwplugin_no_connected_ues_verifier_tests.erl"]}}
and then there is one line for each src folder (I guess), like the one above. Then it repeats it self again and prints all src folders. It prints all src folders 95 times (in total around +23000 lines). At one point there is a 5 seconds gap from one line to the next line (according to time stamp) .... line317: 11:02:22,075 F: (LogEventHandler.java:27) : info: {"/vobs/erbs/int/gte/fw/cpp/test/eunit",{ok,["cpp_fwplugin_no_connected_ues_verifier_tests.erl"]}} .... line567: 11:02:22,279 F: (LogEventHandler.java:27) : info: {"/vobs/erbs/int/gte/fw/cpp/test/eunit",{ok,["cpp_fwplugin_no_connected_ues_verifier_tests.erl"]}}
Then it ends the log with: 11:02:48,865 F: (ErlideBuilder.java:161) : >>> compiler options ::: [debug_info,nowarn_export_all,nowarn_export_vars,nowarn_shadow_vars,warn_unused_function,warn_deprecated_function,nowarn_obsolete_guard,nowarn_unused_import,warn_unused_vars,warn_unused_record] 11:02:48,868 F: (ErlideBuilder.java:167) : Will compile 1 resource(s) 11:02:48,869 F: (Backend.java:241) : backend R16_4fe32_etxalpo_**.ericsson.se: add path /vobs/erbs/int/gte/ebin 11:02:53,783 F: (Backend.java:264) : backend R16_4fe32_etxalpo__.ericsson.se: remove path /vobs/erbs/int/gte/ebin 11:02:53,790 F: (ErlideBuilder.java:264) : ###_* Finished build of INT-GTE took 32149
I found the problem. I added recently code to handle .app.src files and it adds all modules in the project to the list of modules in .app. This is reasonable for a normal project, but in your case it adds all modules in all projects to all .app files that it finds!
I will try to see if I can do something about it, but it is another sign that you should have many small projects instead :-)
yes, I will play with generating .project files and see how that works. thanks for looking into this!