Wrong used size for cache zone

[gleb-shchavlev]

Hello.

I'm having a problem for some Cache Zones the occupied place is not correctly displayed.

zone: xxx
Capacity: 4.00 GiB
Used: 17179869184.00 GiB

zone: yyy
Capacity: 4.00 GiB
Used: 17179869184.00 GiB

All problem zones located at /dev/shm and contains alot of m3u8 files with short period of life (5 sec).

proxy_cache_path /dev/shm/xxx ... inactive=5s ...;

[vozlt]

Hi, Please let me know your nginx version and nginx-module-vts version. Thanks.

[gleb-shchavlev]

nginx version: nginx/1.13.6 nginx-module-vts-0.1.15

[vozlt]

Please let me know detailed cacheZones parts in JSON document. If it is set to a local directory other than /dev/shm, does it look normal?

FYI, The cache informations(maxSize, usedSize) was taken from nginx cache information and nginx-module-vts module just uses it.

$ vi src/ngx_http_vhost_traffic_status_shm.c

232 ngx_http_vhost_traffic_status_shm_add_node_cache(ngx_http_request_t *r,
.
242         c = r->cache;
243         cache = c->file_cache;
.
250         vtsn->stat_cache_max_size = (ngx_atomic_uint_t) (cache->max_size * cache->bsize);
.
254 
255         vtsn->stat_cache_used_size = (ngx_atomic_uint_t) (cache->sh->size * cache->bsize);
.

[daniellockyer]

Hey @vozlt, I also get a weird value for max cache size.

It shows up as 8388608.000000 TiB in the interface. I'll investigate in source code tomorrow.

[gleb-shchavlev]

In json is:

{
	"m3u8": {
		"maxSize": 4294967296,
		"usedSize": 18446744073708896256,
		"inBytes": 8351773681,
		"outBytes": 89223615532,
		"responses": {
			"miss": 17403467,
			"bypass": 0,
			"expired": 16158503,
			"stale": 11478,
			"updating": 219766,
			"revalidated": 615478,
			"hit": 94990725,
			"scarce": 0
		},
		"overCounts": {
			"maxIntegerSize": 18446744073709551615,
			"inBytes": 0,
			"outBytes": 0,
			"miss": 0,
			"bypass": 0,
			"expired": 0,
			"stale": 0,
			"updating": 0,
			"revalidated": 0,
			"hit": 0,
			"scarce": 0
		}
	}
}

In other directories it looks normal.

/dev/shm created by default by centos 7.

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         63G     0   63G   0% /dev
tmpfs            63G  8.7G   55G  14% /dev/shm
tmpfs            63G  4.0G   59G   7% /run
tmpfs            63G     0   63G   0% /sys/fs/cgroup

[vozlt]

@neosilky Did you set max_size in proxy_cache_path directive? If you max_size option do not set in proxy_cache_path directive, it is probably a normal value due to the default assignment value NGX_MAX_OFF_T_VALUE depending on the operating system.

$ vi src/http/ngx_http_file_cache.c

  82 static ngx_int_t
  83 ngx_http_file_cache_init(ngx_shm_zone_t *shm_zone, void *data)
  84 {
.
 155     cache->bsize = ngx_fs_bsize(cache->path->name.data);
 156 
 157     cache->max_size /= cache->bsize;
.
2302 char *
2303 ngx_http_file_cache_set_slot(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
2304 {
.
2343     max_size = NGX_MAX_OFF_T_VALUE;
.
2600     cache->max_size = max_size;

$ vi objs/ngx_auto_config.h

226 #ifndef NGX_MAX_OFF_T_VALUE
227 #define NGX_MAX_OFF_T_VALUE  9223372036854775807LL
228 #endif

[gleb-shchavlev]

@vozlt Hello! Any news? Maybe you need additional information?

[kuroishi]

@gleb-shchavlev What kind of filesystem do you use on proxy_cache_path? I met a similar problem that you have faced with the xfs file system. You might want to use allocsize=4096 as the mount command option.

[gleb-shchavlev]

/dev/shm, tmpfs (rw,nosuid,nodev)

[truedaedalus]

same issue when using cache on ZFS. if using cache dir on ext4 fs, module shows right cache values

nginx version: nginx/1.17.3 nginx-module-vts: latest from git

[vozlt]

@daniellockyer I fixed that maxSize in cacheZones is displayed incorrectly. @gleb-shchavlev Please test it on the latest nginx version((>1.18.0)) and below version.(I did not modify the usedSize part.) Commit: https://github.com/vozlt/nginx-module-vts/commit/549cc4d0d31b8bea5319c8564917875d03f105e1

[vozlt]

@daniellockyer @gleb-shchavlev I have restored it to its original state. I will take a look again. The part I fixed is the size of keys_zone.

vtsn->stat_cache_max_size = (ngx_atomic_uint_t) cache->shm_zone->shm.size;

commit: 203bc5b

[vozlt]

@daniellockyer As I mentioned above, if max_size in proxy_cache_path directive is not specified, the system dependent value NGX_MAX_OFF_T_VALUE is assigned by default. I added additional information about cacheZones. Commit: https://github.com/vozlt/nginx-module-vts/commit/91bdb146e45e612d1ce0b257bcda638848367d84