fluentd-docker-image icon indicating copy to clipboard operation
fluentd-docker-image copied to clipboard

[alpine images] Coolio::StatWatcher fallbacks to polling instead of using inotify

Open adw1n opened this issue 7 years ago • 1 comments

Description

When I use the fluent/fluentd:v1.1.0 image the in_tail plugin fallbacks to polling file every 5 seconds instead of using inotify. This causes loss in timestamp precision. fluent/fluentd:v1.1.0-debian image handles watching file correctly and does not fallback to polling.

Steps to reproduce

  1. put Dockerfile, fluent.conf and write_logs.sh in one directory Dockerfile
FROM fluent/fluentd:v1.1.0

COPY ./fluent.conf /fluentd/etc/fluent.conf
COPY ./write_logs.sh /usr/local/bin/write-logs
RUN chmod 755 /usr/local/bin/write-logs

RUN find . -name in_tail.rb | xargs sed -i '/line.chomp!/a \ \ \ \ \ \ \ \ puts Time.now.to_datetime.iso8601(3)'

fluent.conf

<source>
  @type tail
  path /tmp/test.log
  pos_file /tmp/test.log.pos
  tag test
  path_key tailed_test
  keep_time_key true
  format none
  enable_watch_timer false
</source>

write_logs.sh

#!/bin/sh
counter=0
while true; do
	sleep 0.01
	echo $counter >> /tmp/test.log
	counter=`expr $counter + 1`
done
  1. run docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
  2. run docker exec -it inotify_problem write-logs

As you can see, logs are being polled with 5 sec frequency (convert_line_to_event function is executed every ~5 sec).

Now change fluent/fluentd:v1.1.0 to fluent/fluentd:v1.1.0-debian in the Dockerfile and run:

  1. docker stop inotify_problem; docker rm inotify_problem; docker rmi inotify_problem
  2. docker build -t inotify_problem . && docker run --name inotify_problem -it inotify_problem
  3. docker exec -it inotify_problem write-logs

Observe that convert_line_to_event function is executed every ~0.01 sec.

adw1n avatar Feb 19 '18 13:02 adw1n

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 7 days

github-actions[bot] avatar Aug 29 '24 11:08 github-actions[bot]

This issue was automatically closed because of stale in 7 days

github-actions[bot] avatar Dec 01 '24 11:12 github-actions[bot]