what is fragmentation and how it influences performance and stability?

[mente]

We're running an experiment of upgrading php5 to php 7.0.12 and apcu 5.1.6 from 4.0.7.

There's a big increase in apc items being stored and potentially fragmentation reaching 100% at some point of time for php7 servers, while php5 is all stable.

Question is in title: what is fragmentation and what possibly can be related?

php5 configuration:

$ php5 -i  | fgrep apc
/etc/php5/cli/conf.d/20-apcu.ini,
apc
apcu
apc.coredump_unmap => Off => Off
apc.enable_cli => Off => Off
apc.enabled => On => On
apc.entries_hint => 16384 => 16384
apc.gc_ttl => 3600 => 3600
apc.mmap_file_mask => /dev/zero => /dev/zero
apc.preload_path => no value => no value
apc.rfc1867 => Off => Off
apc.rfc1867_freq => 0 => 0
apc.rfc1867_name => APC_UPLOAD_PROGRESS => APC_UPLOAD_PROGRESS
apc.rfc1867_prefix => upload_ => upload_
apc.rfc1867_ttl => 3600 => 3600
apc.serializer => igbinary => igbinary
apc.shm_segments => 1 => 1
apc.shm_size => 1000M => 1000M
apc.slam_defense => Off => Off
apc.smart => 0 => 0
apc.ttl => 3600 => 3600
apc.use_request_time => On => On
apc.writable => /run => /run

php7 configuration:

$ php -i | fgrep apc
/etc/php/7.0/cli/conf.d/20-apcu.ini,
/etc/php/7.0/cli/conf.d/20-apcu_bc.ini,
apc
apcu
apc.coredump_unmap => Off => Off
apc.enable_cli => Off => Off
apc.enabled => On => On
apc.entries_hint => 131072 => 131072
apc.gc_ttl => 3600 => 3600
apc.mmap_file_mask => /dev/zero => /dev/zero
apc.preload_path => no value => no value
apc.serializer => php => php
apc.shm_segments => 1 => 1
apc.shm_size => 1000M => 1000M
apc.slam_defense => Off => Off
apc.smart => 0 => 0
apc.ttl => 3600 => 3600
apc.use_request_time => On => On
apc.writable => /run => /run

There's only difference in apc on php5 and php7 - serializer is php and not igbinary because igbinary somehow results in zend_mm_heap corrupt but this is another problem.

[krakjoe]

Wait for the next release, test that please.

[mente]

So I've tried latest release - issue is the same. Actually we have the same issue both in php5.6 with apcu 4.0 and in php7 with 5.1.7 version. But guess I've found the problem with our configuration:

apc.ttl = 3600 and apc.gc_ttl = 3600 .

It looks ok unless check the actual usage of apcu in our system. It is used for doctrine query/result cache in symfony. And default ttl for apcu cache is 0 there, meaning cache forever. But with these apc.ttl and apc.gc_ttl settings it's never evicted, thus fragmentation increases.

And here is a apcu_cache_info with 3 days running fpm and fragmentation 100%:

array(10) {
  ["num_slots"]=>
  int(16411)
  ["ttl"]=>
  int(3600)
  ["num_hits"]=>
  float(266164069)
  ["num_misses"]=>
  float(31399893)
  ["num_inserts"]=>
  float(12428796)
  ["num_entries"]=>
  int(79643)
  ["expunges"]=>
  float(0)
  ["start_time"]=>
  int(1477584838)
  ["mem_size"]=>
  float(369805424)
  ["memory_type"]=>
  string(4) "mmap"
}

So the question is: how can I make apc flush entries even if entry's TTL is 0?

[dugwood]

You can run a script to remove keys that are too far in the future (beware of the memory_limit with apc_cache_info, I now suggest to use APCIterator), by looking at each key:

Array (
 [info] => your_key
 [ttl] => 3600
 [num_hits] => 26
 [modification_time] => 1492668617
 [creation_time] => 1492668617
 [deletion_time] => 0
 [access_time] => 1492669074
 [ref_count] => 0
 [mem_size] => 656
)

If ttl is zero, then check the modification_time to decide if you need to remove it or not.

I suggest you try to fix the issue in Doctrine, that's a better idea.

You can also prefix all your keys with a «release version», and when you're upgrading version, you just remove old keys. For example, if you have a prefix my-prefix-1234- and you upgrade to 1235, then you iterate all keys starting with my-prefix-*-, and if the * isn't 1235, you can delete it with apc_delete.

About the fragmentation, I wrote a comment: https://github.com/krakjoe/apcu/issues/127#issuecomment-244362339 I really don't think fragmentation is a big issue, if you have enough free space and keys that expire within a day or so. In your case and no expire keys, you will clearly have an issue :-)

[williamdes]

still open ?