Interrupted processing

[geoffroy-noel-ddh]

We are using Open layer to display JP2 images via the Zoomify protocol. On the server side we have Apache2 and the latest version of IIPSrv (compiled just a few days ago from github) through fcgi.

Most of the times the zoomify tiles display correctly in the browser. But from time to time OpenLayer will show broken tiles/images. A typical scenario is that all four tiles fail. One with a 500 error and the three others with net::ERR_EMPTY_RESPONSE (in Chrome). I paste the corresponding log below.

All other non iipsrv requests made to Apache by the web page always return a valid response. This is why I'm thinking the issue might be with IIPSrv (or the way it is configured).

In the log below you can see the 4 requests. The first two and the fourth are terminated, The second (which returns a 500 error tot he browser) is probably crashing while handling the tiles. Note that at the time of execution I had two IIPSrv processes running in parallel.

Have you seen this before? I'm quite puzzled by the terminated requests and wondering if the cause is within IIPSrv or in Apache.

Thanks in advance for any hint about what's going on here.

Caught Terminated signal. Terminating after 2 accesses
Mon Aug 17 23:42:01 2015
<----------------------------------->


Caught Terminated signal. Terminating after 2 accesses
Mon Aug 17 23:42:01 2015
<----------------------------------->

<----------------------------------->
Mon Aug 17 23:42:07 2015

IIPImage Server. Version 1.0
*** Ruven Pillay <[email protected]> ***

Verbosity level set to 10
Running in FCGI mode

Setting maximum image cache size to 256MB
Setting filesystem prefix to '/vol/[...]'
Setting default JPEG quality to 75
Setting maximum CVT size to 5000
Setting HTTP Cache-Control header to 'max-age=86400'
Setting 3D file sequence name pattern to '_pyr_'
Setting max quality layers (for supported file formats) to 10
Setting up JPEG2000 support via Kakadu SDK


Initialisation Complete.
<----------------------------------->

Full Request is zoomify=jp2/test.jp2/TileGroup0/1-1-0.jpg
[1/1]: Command / Argument is zoomify : jp2/test.jp2/TileGroup0/1-1-0.jpg
Zoomify handler reached
FIF handler reached
FIF :: URL decoding/filtering: jp2/test.jp2 => jp2/test.jp2
FIF :: Image cache initialization
FIF :: JPEG2000 image detected
FIF :: Created image
FIF :: Image dimensions are 3666 x 3479
FIF :: Image contains 3 channels with 8 bits per channel
FIF :: Image timestamp: Tue, 04 Nov 2014 15:16:21 GMT
FIF :: Total command time 508 microseconds
Zoomify :: Tile request for resolution:1 at x:1, y:0
JTL handler reached
TileManager :: Cache Miss for resolution: 1, tile: 1
TileManager :: Cache Size: 0 tiles, 0 MB

Caught Terminated signal. Terminating after 0 accesses
Mon Aug 17 23:42:07 2015
<----------------------------------->

[ruven]

Is there anything about this in the Apache error log file? Does this still occur if you have just 1 iipsrv process? You could also try starting iipsrv on the command line rather than have Apache start it for you and see if there are any messages.

[geoffroy-noel-ddh]

Yes, it also occurs with a single fcgid process. Here are the apache logs.

[Thu Aug 20 14:25:28 2015] [error] [client XXX] Premature end of script headers: iipsrv.2015-08-13.fcgi, referer: https://xxx/digipal/search/facets/?page=1&result_type=images&img_is_public=1&view=grid

[Thu Aug 20 14:25:28 2015] [emerg] mod_fcgid: server is restarted, pid 2537 must exit
[Thu Aug 20 14:25:28 2015] [emerg] mod_fcgid: server is restarted, pid 2537 must exit
[Thu Aug 20 14:25:29 2015] [error] mod_fcgid: fcgid process manager died, restarting the server
[Thu Aug 20 14:25:29 2015] [notice] SIGHUP received.  Attempting to restart

I can reproduce it systematically. But I have two cases: when I do a normal page load (where most resources are cached by the browser) the image will often display properly. When I do a hard-refresh I almost always have a 500 server error from apache with the log as copied above. So it seems that the situation is different when apache has to deal with more requests coming in. Hence the difficulty to create a test case I could sen to you.

When I get a 500, the last line in the IIPSrv log is always when related to the TileManager.

I'm still unsure whether this is a crash in IIPSrv or something above that makes the process stop. I'll try running the service from the command line to see what happens.