No logs in Kibana

[mohamedhaleem]

Hi

i am trying to use your project to collect logs from our docker containers

I start the container as follows:

docker run -d -v /var/run/docker.sock:/tmp/docker.sock -e LOGSTASH_HOST=96.x.x.A -e LOGSTASH_PORT=5044 --name filebeat bargenson/filebeat

I can load up kibana UI but i can't see any logs .. so i tried the following from the machine running docker-filebeat:

nc -w1 96.x.x.A 5044 <<< "testing again from my home machine"

And the machine having the log stash forwarder, i see the following...

{:timestamp=>"2016-03-09T03:30:18.611000+0000", :message=>"Beats Input: Remote connection closed", :peer=>"96.x.x.B:58768", :exception=>#<Lumberjack::Beats::Connection::ConnectionClosed: Lumberjack::Beats::Connection::ConnectionClosed wrapping: Lumberjack::Beats::Parser::UnsupportedProtocol, unsupported protocol 116>, :level=>:warn}

How to resolve this?

[mohamedhaleem]

hi again

Thank you for making this project - now i am getting logs in Kibana UI, but only the logs from this container itself - and only if i execute filebeat in debug mode. Nothing from the other containers. I checked logs are being emitted by the other containers

Does anyone else have this working?

[bargenson]

Hi Mohamed,

We've been using that image on different production environment for weeks and we didn't have that kind of issue. Could you share your filbeat container's logs? You should have entries like

Processing $CONTAINER...

or

Disconnected from $CONTAINER.

[Ryuzavi]

I'll piggy back on this one. I'm having a similar issue trying to hook up this container to the sebp/elk container.

All I get from the logs is the following repeated constantly:

7/6/2016 3:27:42 PMDisconnected from 583f22e408b39b7b9102b29887d0f2620029c3d4a0d7b65a9fb040ea8fdaaa43.
7/6/2016 3:27:47 PM2016/07/06 14:27:47.671613 single.go:76: INFO Error publishing events (retrying): EOF
7/6/2016 3:27:47 PM2016/07/06 14:27:47.671643 single.go:152: INFO send fail
7/6/2016 3:27:47 PM2016/07/06 14:27:47.671658 single.go:159: INFO backoff retry: 1m0s

I have added the cert suggested by sebp in the docs here but this doesn't seem to solve the issue.

Any advice?

[rizwan707]

i dont see any logs in kibana i am running this image can you please help me

[rizwan707]

input {
beats { port => "5044" } }

filter {

if [type] == "filebeat-docker-logs" {

grok {
  match => { 
    "message" => "\[%{WORD:containerName}\] %{GREEDYDATA:message_remainder}"
  }
}

mutate {
  replace => { "message" => "%{message_remainder}" }
}

mutate {
  remove_field => [ "message_remainder" ]
}

}

}

output {
elasticsearch { hosts => ["localhost:9200"] } }

this is my logstash config

[Altenrion]

@rizwan707 have yu solved your issue with merging this two containerised projects?