fluentd-docs-gitbook icon indicating copy to clipboard operation
fluentd-docs-gitbook copied to clipboard

Published 20 hours ago • verified publisher icon fluent

Docker-compose example does not work correctly

Open ozang opened this issue 3 years ago • 5 comments

Hi, docker-compose example in page https://docs.fluentd.org/container-deployment/docker-compose does not work correctly. Kibana shows only shows errors saying "there is a tag error." I cannot see any access log related web access.

ozang avatar Apr 02 '21 11:04 ozang

image

Hmm, it seems that it just works for me. Does httpd image up and running correctly?

kenhys avatar Apr 12 '21 07:04 kenhys

Actually the latest image in the page https://docs.fluentd.org/container-deployment/docker-compose shows the error's reason. I mean shared picture already says the reason, It seems to be a config problem.

ozang avatar Apr 12 '21 07:04 ozang

I'm too was trying to get that example running. There several things off between the tutorial and the actual result.

  • The web service won't return <html><body><h1>It works!</h1></body></html> without any configuration, and none is provided.
  • The tag httpd.access won't appear in kibana, that's something I was missing and led me to this issue.
  • fluent.conf in docs and in this repo differ, see below.

From my experiments, I can say that:

FROM fluent/fluentd:v1.14-1

USER root

RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.1.4"]

updating fluentd and elasticsearch

and -- maybe more importantly -- omitting the <label @FLUENT_LOG> part from fluent.conf fixed the issue for me. Note that the conf file change is a key difference between the tutorial code and code in this repo. This also was the key difference between a working and nonworking solution for me.

With it, as @ozang reported, I'm getting the tag error, and this is the only tagged message in ELK later on. It appears that with it, fluentd won't recognize the "match *.**" clause as mathing for that tag.

I have minor docker-compose.yml changes locally because of my project but I think these are not relevant. @ozang, maybe you want to try with a current fluentd and leaving out the label in the fluentd configuration.

If there is interest, I volunteer to rewalk the example and create a PR. But please consider, I'm absolutely new to fluentd so I don't know 100% what I'll be doing.

sebkraemer avatar Dec 22 '21 21:12 sebkraemer

Actually the latest image in the page https://docs.fluentd.org/container-deployment/docker-compose shows the error's reason. I mean shared picture already says the reason, It seems to be a config problem.

Actually, the image of that page says no patterns matched...

  • https://github.com/fluent/fluentd-docs-gitbook/blob/1.0/.gitbook/assets/7.10_efk-kibana-discover.png

I don't know the situation, but there is no problem with the Fluentd config on that page, right?

and -- maybe more importantly -- omitting the <label @FLUENT_LOG> part from fluent.conf fixed the issue for me.

This is for the config on the following repository, right?

https://github.com/digikin/fluentd-elastic-kibana

There are other reports of the information on the page not working. We need to fix it.

If there is interest, I volunteer to rewalk the example and create a PR. But please consider, I'm absolutely new to fluentd so I don't know 100% what I'll be doing.

Sorry for the delay. If it is still possible, I would appreciate it if you could try to make a PR.

Note that we need to update the 1.0 branch. We will review it, so no worries!

daipom avatar Apr 20 '23 03:04 daipom

Sorry for the delay. If it is still possible, I would appreciate it if you could try to make a PR.

Hi @daipom, sorry for getting back to you so late and with bad news. But I used the setup only for a small POC during a learning session and I have that setup or could get it up (and fixed properly) easily. Haven't used fluentd since..

sebkraemer avatar Dec 22 '23 16:12 sebkraemer