container memory increasing after deploy mtail using docker

[ruyifu159]

Current mtail version: 3.0.0-rc47.

create mtail container's command: docker run -d -p 3803:3803 -m 2048m --name mtail -v /data/xx/workspace/mtail:/etc/mtail -v /data/prelog:/var/log/xx dylanmei/mtail:v3.0.0-rc47 -logtostderr --port 3803 --expired_metrics_gc_interval 30m --stale_log_gc_interval 10m --vm_logs_runtime_errors=false --progs /etc/mtail --logs '/var/log/xx/*.log'

Next, my mtail container running and configure mtail job exporter and telegraf job exporter in prometheus. Through prometheus's monitor panel using command docker_container_mem_usage_percent{container_name="mtail"}, I find mtail container's memory increasing, follow as:

[jaqx0r]

https://google.github.io/mtail/Troubleshooting.html#memory-or-performance-issues has some tips on debugging where memory usage is occuring.

If the program is not releasing metrics then the expiration gc time won't affect any memory.

Please share the mtail program so we can understand what it is doing.

On Thu, 9 Jun 2022 at 14:50, 付如意 @.***> wrote:

Current mtail version: 3.0.0-rc47.

create mtail container's command: docker run -d -p 3803:3803 -m 2048m --name mtail -v /data/xx/workspace/mtail:/etc/mtail -v /data/prelog:/var/log/xx dylanmei/mtail:v3.0.0-rc47 -logtostderr --port 3803 --expired_metrics_gc_interval 30m --stale_log_gc_interval 10m --vm_logs_runtime_errors=false --progs /etc/mtail --logs '/var/log/xx/*.log'

Next, my mtail container running and configure mtail job exporter and telegraf job exporter in prometheus. Through prometheus's monitor panel using command docker_container_mem_usage_percent{container_name="mtail"}, I find mtail container's memory increasing, follow as: [image: CE378A44-31C5-41a9-B772-F3FE77967E9B] https://user-images.githubusercontent.com/5111004/172766248-f64092dc-cb93-4728-92b1-48b8ff3743f6.png

— Reply to this email directly, view it on GitHub https://github.com/google/mtail/issues/650, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFX65ZWJICGJ3CXWD2OE3VOFZYFANCNFSM5YIWBVOQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ruyifu159]

Do you want to know mtail regular file？ Actualy, I write some .mtail regular statement, and they has similar syntax, for example:

gauge controller_resp_time by name, param

/<(?P<name>.*)> 执行:(?P<time>.*)ms  响应:(?P<param>.*)/ {
	controller_resp_time[$name][$param] = $time
}

so I guess that similar these mtail statement is too grow. Because I find mtail provide two args, follow as:

 -max_recursion_depth int
    	The maximum length a mtail statement can be, as measured by parsed tokens. Excessively long mtail expressions are likely to cause compilation and runtime performance problems. (default 100)
  -max_regexp_length int
    	The maximum length a mtail regexp expression can have. Excessively long patterns are likely to cause compilation and runtime performance problems. (default 1024)

And it indeed to describe that these are likely to cause compilation and runtime performance problems.

https://google.github.io/mtail/Troubleshooting.html#memory-or-performance-issues has some tips on debugging where memory usage is occuring. If the program is not releasing metrics then the expiration gc time won't affect any memory. Please share the mtail program so we can understand what it is doing. … On Thu, 9 Jun 2022 at 14:50, 付如意 @.> wrote: Current mtail version: 3.0.0-rc47. create mtail container's command: docker run -d -p 3803:3803 -m 2048m --name mtail -v /data/xx/workspace/mtail:/etc/mtail -v /data/prelog:/var/log/xx dylanmei/mtail:v3.0.0-rc47 -logtostderr --port 3803 --expired_metrics_gc_interval 30m --stale_log_gc_interval 10m --vm_logs_runtime_errors=false --progs /etc/mtail --logs '/var/log/xx/.log' Next, my mtail container running and configure mtail job exporter and telegraf job exporter in prometheus. Through prometheus's monitor panel using command docker_container_mem_usage_percent{container_name="mtail"}, I find mtail container's memory increasing, follow as: [image: CE378A44-31C5-41a9-B772-F3FE77967E9B] https://user-images.githubusercontent.com/5111004/172766248-f64092dc-cb93-4728-92b1-48b8ff3743f6.png — Reply to this email directly, view it on GitHub <#650>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFX65ZWJICGJ3CXWD2OE3VOFZYFANCNFSM5YIWBVOQ . You are receiving this because you are subscribed to this thread.Message ID: @.*>