Website cannot run (FATAL:shared_mem_cache.cc(564)] Check failed: worker != NULL)

[chaihongjun]

here is the nginx error log: 2018/09/10 17:10:54 [alert] 2282#0: [ngx_pagespeed 1.13.35.2-0] [0910/171054:FATAL:shared_mem_cache.cc(564)] Check failed: worker != NULL. nginx: worker process() [0x56a86e] nginx: worker process() [0x4f9a12] nginx: worker process() [0x568ec0] nginx: worker process() [0x6f9a12] nginx: worker process() [0x6fe3d5] nginx: worker process() [0x6fe6ec] nginx: worker process() [0x661409] nginx: worker process() [0x663cef] nginx: worker process() [0x663416] nginx: worker process() [0x661409] nginx: worker process() [0x6f2223] nginx: worker process() [0x6f5816] nginx: worker process() [0x6f24c6] nginx: worker process() [0x8d4d2d] nginx: worker process() [0x59aee3] nginx: worker process() [0x8a383d] nginx: worker process() [0x6a4d95] nginx: worker process() [0x58ebac] nginx: worker process() [0x6a25dc] nginx: worker process() [0x58e835] nginx: worker process() [0x564c90] nginx: worker process() [0x644ff7] nginx: worker process() [0x70d2f0] nginx: worker process() [0x644ff7] nginx: worker process() [0x7103d7] nginx: worker process() [0x712292] /lib64/libpthread.so.0() [0x3c80c07aa1] /lib64/libc.so.6(clone+0x6d) [0x3c808e8bcd]

[oschaaf]

It's this check that is failing: https://github.com/apache/incubator-pagespeed-mod/blob/master/pagespeed/kernel/sharedmem/shared_mem_cache.cc#L566

Can you post the anonymized (pagespeed) configuration?

[chaihongjun]

@oschaaf

pagespeed on; pagespeed InPlaceResourceOptimization on; pagespeed EnableFilters in_place_optimize_for_browser; pagespeed RespectVary on; pagespeed LowercaseHtmlNames on; pagespeed LazyloadImagesAfterOnload on; pagespeed EnableFilters inline_preview_images; pagespeed EnableFilters hint_preload_subresources;
pagespeed FetchHttps enable,allow_self_signed,allow_unknown_certificate_authority,allow_certificate_not_yet_valid; pagespeed SslCertDirectory /etc/letsencrypt/acme; pagespeed SslCertFile /etc/letsencrypt/acme/fullchain.cer; pagespeed ReportUnloadTime on; pagespeed LoadFromFile "https://pc.example.com/static/css" "/alidata1/www/web/example.com/www.example.com/static/css"; pagespeed LoadFromFile "https://pc.example.com/static/images" "/alidata1/www/web/example.com/www.example.com/static/images"; pagespeed LoadFromFile "https://pc.example.com/static/img" "/alidata1/www/web/example.com/www.example.com/static/img"; pagespeed LoadFromFile "https://pc.example.com/static/js" "/alidata1/www/web/example.com/www.example.com/static/js"; pagespeed LoadFromFile "https://pc.example.com/uploads" "/alidata1/www/web/example.com/www.example.com/uploads"; pagespeed LoadFromFileRuleMatch allow .css$; pagespeed LoadFromFileRuleMatch allow .jpe?g$; pagespeed LoadFromFileRuleMatch allow .png$; pagespeed LoadFromFileRuleMatch allow .gif$; pagespeed LoadFromFileRuleMatch allow .js$; pagespeed LoadFromFileRuleMatch allow .webp$; pagespeed FileCachePath /dev/shm/ngx_pagespeed_cache/pc.example.com;
pagespeed FileCacheCleanIntervalMs -1; pagespeed FileCacheInodeLimit 500000; pagespeed RetainComment " Make "; pagespeed EnableFilters prioritize_critical_css; pagespeed EnableFilters flatten_css_imports; pagespeed CssFlattenMaxBytes 5120; pagespeed EnableFilters add_head; pagespeed DisableFilters add_instrumentation; pagespeed EnableFilters combine_heads; pagespeed EnableFilters convert_meta_tags; pagespeed EnableFilters dedup_inlined_images; pagespeed PrivateNotVaryForIE off; pagespeed EnableFilters inline_images; pagespeed EnableFilters recompress_images; pagespeed EnableFilters jpeg_subsampling; pagespeed EnableFilters convert_gif_to_png; pagespeed EnableFilters convert_png_to_jpeg; pagespeed EnableFilters convert_jpeg_to_progressive; pagespeed EnableFilters convert_to_webp_lossless; pagespeed EnableFilters convert_to_webp_animated; pagespeed EnableFilters inline_import_to_link; pagespeed EnableFilters local_storage_cache; pagespeed EnableFilters move_css_above_scripts; pagespeed EnableFilters move_css_to_head; pagespeed RewriteLevel PassThrough; pagespeed EnableFilters collapse_whitespace;
pagespeed EnableFilters combine_css;
pagespeed EnableFilters combine_javascript;
pagespeed EnableFilters rewrite_css,sprite_images; pagespeed EnableFilters elide_attributes;
pagespeed EnableFilters extend_cache;
pagespeed EnableFilters lazyload_images; pagespeed EnableFilters responsive_images_zoom; pagespeed EnableFilters trim_urls;
pagespeed EnableFilters rewrite_javascript;
pagespeed UseExperimentalJsMinifier on; pagespeed EnableFilters strip_image_meta_data; pagespeed EnableFilters insert_dns_prefetch; pagespeed EnableFilters remove_comments; pagespeed EnableFilters trim_urls; pagespeed EnableFilters rewrite_style_attributes_with_url; pagespeed EnableFilters make_google_analytics_async; pagespeed Disallow "/pma/"; pagespeed Disallow "/include/*"; pagespeed Disallow .pagespeed.; pagespeed XHeaderValue "Desktop Proudly Presented by Google"; pagespeed PreserveUrlRelativity off; pagespeed EnableFilters make_google_analytics_async; pagespeed SupportNoScriptEnabled false; pagespeed Statistics on; pagespeed StatisticsLogging on; pagespeed StatisticsLoggingIntervalMs 60000; pagespeed StatisticsLoggingMaxFileSizeKb 1024; pagespeed LogDir /alidata1/pagespeed_logs; pagespeed StatisticsPath /ngx_pagespeed_statistics; pagespeed MessagesPath /ngx_pagespeed_message; pagespeed ConsolePath /pagespeed_console; pagespeed AdminPath /pagespeed_admin; pagespeed EnableCachePurge off; pagespeed Domain https://pc.example.com; location ~ ".pagespeed.([a-z].)?[a-z]{2}.[^.]{10}.[^.]+" {add_header "" "";} location ~ "^/ngx_pagespeed_static/" {allow 127.0.0.1; deny all;} location /ngx_pagespeed_message {allow 127.0.0.1; deny all;} location ~ "^/ngx_pagespeed_beacon$" { } location /pagespeed_console { } location ~ ^/pagespeed_admin { }

[oschaaf]

Thanks! Hmm, are there any other warnings / errors in the logs? I wonder, does using another path here resolve the issue?:

pagespeed FileCachePath /dev/shm/ngx_pagespeed_cache/pc.example.com;

Can you correlate the timing of the issue to a particular event (E.g. nginx restarting a worker?).

[chaihongjun]

when i changed some pagepseed config, then restart the nginx, and the nginx reload success but website cannot run

[oschaaf]

Do you remember what you changed?

[chaihongjun]

just "pagespeed LoadFromFile"

[chaihongjun]

[oschaaf]

So, if I understand correctly, before adding LoadFromFile lines, the problem did not occur?

[chaihongjun]

no ,this has happened before i think it does't matter with config

[oschaaf]

So I haven't been able to reproduce this, would you be able to post a gdb backtrace? That is probably going to greatly speed up diagnosing this

[chaihongjun]

I have resolved this problem. you may feel amazing that i remove all the others config section,then everythings is ok. so i guest the incorrect comment is the answer

[oschaaf]

Do you know which specific line(s) triggered this issue?

[chaihongjun]

Sorry, I am not accurate enough to express the question. I mean i have another websites winthin the same server ,that i have placed pagespeed configs in these conf files directly.Some of these configuration items have comments.And I have removed all of these items and comments .Then everythins is ok. Maybe there are some mistakes in my way of writing notes? I will add some pagespeed items and see if there is any problem

[chaihongjun]

these pagespeed config are all the same.

[hjf288]

Do you know which specific line(s) triggered this issue?

I've seen the same behavior, it appears to occur if I have pagespeed FileCachePath defined in a server block but not the HTTP block.

Setting pagespeed ShmMetadataCacheCheckpointIntervalSec to-1 in the HTTP block or defining the FileCachePath before it seems to prevent this from occurring.