Hale Studio freeze while WFS Import

[nicolasdiedrich]

Hale studio freezes when importing WFS data.

• Occurs after the occurrence of an error during WFS import when import is repeated.
• The process continues endlessly and aborting is not possible.
• An error message does not occur again
• Only solution: shut down Hale studio via task manager

[dhdeangelis]

I wonder if this issue is solved or any ideas have emerged for a workaround. I am using Hale 4.1.0 (released one week ago) and I am experiencing a similar problem with one particular WFS service: the GUI stalls while loading the source data. Even a small subset, 100-500 features, cannot finish loading. This particular WFS exposes a total of 1.2 million features but I am not loading all of it at once of course. Is this perhaps related to paging issues? This service works OK in for example QGIS. However, tests with another service (exposing around 85000 features) does not show this problem. IS this possibly showing problems with the WFS service itself?

Thoughts appreciated

/Hernán

With a small subset I can get the following error log in a reasonable time (several minutes):

java.lang.IllegalStateException: Could not open instance input at eu.esdihumboldt.hale.io.gml.reader.internal.GmlInstanceCollection$GmlInstanceIterator.<init>(GmlInstanceCollection.java:125) at eu.esdihumboldt.hale.io.gml.reader.internal.GmlInstanceCollection.iterator(GmlInstanceCollection.java:736) at eu.esdihumboldt.hale.io.gml.reader.internal.wfs.WfsBackedGmlInstanceCollection$WfsBackedGmlInstanceIterator.createNextIterator(WfsBackedGmlInstanceCollection.java:421) at eu.esdihumboldt.hale.io.gml.reader.internal.wfs.WfsBackedGmlInstanceCollection$WfsBackedGmlInstanceIterator.<init>(WfsBackedGmlInstanceCollection.java:384) at eu.esdihumboldt.hale.io.gml.reader.internal.wfs.WfsBackedGmlInstanceCollection.iterator(WfsBackedGmlInstanceCollection.java:266) at eu.esdihumboldt.hale.io.gml.reader.internal.wfs.WfsBackedGmlInstanceCollection.iterator(WfsBackedGmlInstanceCollection.java:1) at eu.esdihumboldt.hale.ui.io.instance.InstanceImportAdvisor.handleResults(InstanceImportAdvisor.java:81) at eu.esdihumboldt.hale.ui.io.instance.InstanceImportAdvisor.handleResults(InstanceImportAdvisor.java:1) at eu.esdihumboldt.hale.ui.io.IOWizard$2.run(IOWizard.java:658) at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122) Caused by: java.lang.OutOfMemoryError: Java heap space at com.google.common.io.ByteStreams.toByteArrayInternal(ByteStreams.java:176) at com.google.common.io.ByteStreams.toByteArray(ByteStreams.java:221) at eu.esdihumboldt.hale.common.cache.Request.get(Request.java:227) at eu.esdihumboldt.hale.common.core.io.supplier.DefaultInputSupplier.resolve(DefaultInputSupplier.java:88) at eu.esdihumboldt.hale.common.core.io.supplier.DefaultInputSupplier.getInput(DefaultInputSupplier.java:60) at eu.esdihumboldt.hale.common.core.io.supplier.DefaultInputSupplier.getInput(DefaultInputSupplier.java:1) at eu.esdihumboldt.hale.io.gml.reader.internal.GmlInstanceCollection$GmlInstanceIterator.<init>(GmlInstanceCollection.java:122) ... 9 more

[thorsten-reitz]

@florianesser Might this be related to the problem where, due to incorrect paging, the same features were loaded again and again? The stacktrace looks different though.

[dhdeangelis]

Thanks for your comment, @thorsten-reitz I must correct my post above, I see the problem with both WFS services, not with only one as implied above.

[dhdeangelis]

I found a workaround to this. I set the project up in the GUI using a WFS GetFeatures request with a low Count limit (100). After the alignment was done I called Hale from the command line using a WFS adress without "Count" parameter as data source. No delays or endless loops.

[JohannaOtt]

@dhdeangelis I would like to check if this behavior is still occurring. Do you still have the WFS available for which this happened to you? If yes, would you be able to share the URL?

[dhdeangelis]

@JohannaOtt thanks for your interest. I finished this particular project almost 2 years ago. I can see now that the problem is not present in the current Hale version I am using (5.0.1), with 15 Gb cache (as stated in HALE.ini). The WFS address is https://epub.sjv.se/inspire/wfs?, the layer LU.jordbruksblock has 1275181 features, of which 1 million load OK, albeit slowly of course. I have not tested enough with paging settings to see whether or not I can bypass the 1 million server side limit using the GUI.

[JohannaOtt]

@dhdeangelis Thank you for the input. I was able to load the full layer with paging enabled (10.000 objects per page). I therefore close this issue now. In case the problem is occurring again, it can be re-opened.