fluent-plugin-geoip
fluent-plugin-geoip copied to clipboard
EFK missing geo_point
Problem
I am running EFK using ECK 8.5.3. fluentd ConfigMap:
@type geoip
# Specify one or more geoip lookup field which has ip address (default: host)
geoip_lookup_keys IP
# Specify optional geoip database (using bundled GeoLiteCity databse by default)
# geoip_database "/path/to/your/GeoIPCity.dat"
# Specify optional geoip2 database
# geoip2_database "/path/to/your/GeoLite2-City.mmdb" (using bundled GeoLite2-City.mmdb by default)
# Specify backend library (geoip2_c, geoip, geoip2_compat)
backend_library geoip2_c
# Set adding field with placeholder (more than one settings are required.)
<record>
city ${city.names.en["IP"]}
latitude ${location.latitude["IP"]}
longitude ${location.longitude["IP"]}
country_code ${country.iso_code["IP"]}
country_name ${country.names.en["IP"]}
postal_code ${postal.code["IP"]}
location_properties '{ "lat" : ${location.latitude["IP"]}, "lon" : ${location.longitude["IP"]} }'
location_string ${location.latitude["IP"]},${location.longitude["IP"]}
location_array '[${location.longitude["IP"]},${location.latitude["IP"]}]'
</record>
ES template:
"mappings": {
"properties": {
"location_properties": { "type": "geo_point" },
"location_string": { "type": "geo_point" },
"location_array": { "type": "geo_point" }
}
}
I don't see any of the properties in Kibana ECK 8.5.3 at all. What do I miss? ...
Steps to replicate
Provide example config and message
Expected Behavior
...
Your environment
-
OS version Ubuntu 22.10
-
paste result of
fluentd --versionortd-agent --versiontd-agent 4.4.2 fluentd 1.15.3 (e89092ce1132a933c12bb23fe8c9323c07ca81f5) -
plugin version
- paste boot log of fluentd or td-agent
- paste result of
fluent-gem list,td-agent-gem listor your Gemfile.lock
*** LOCAL GEMS ***
abbrev (default: 0.1.0)
addressable (2.8.1)
async (1.30.3)
async-http (0.59.2)
async-io (1.34.0)
async-pool (0.3.12)
aws-eventstream (1.2.0)
aws-partitions (1.650.0)
aws-sdk-core (3.164.0)
aws-sdk-kms (1.58.0)
aws-sdk-s3 (1.116.0)
aws-sdk-sqs (1.51.1)
aws-sigv4 (1.5.2)
base64 (default: 0.1.1)
benchmark (default: 0.2.0)
bigdecimal (default: 3.1.1)
bindata (2.4.14)
bundler (2.3.18, default: 2.3.7)
cgi (default: 0.3.1)
cmetrics (0.3.3)
concurrent-ruby (1.1.10)
console (1.16.2)
cool.io (1.7.1)
csv (default: 3.2.2)
date (default: 3.2.2)
debug (1.4.0)
delegate (default: 0.2.0)
did_you_mean (default: 1.6.1)
dig_rb (1.0.1)
digest (default: 3.1.0)
digest-crc (0.6.4)
digest-murmurhash (1.1.1)
drb (default: 2.1.0)
elastic-transport (8.1.0)
elasticsearch (8.4.0)
elasticsearch-api (8.4.0)
english (default: 0.7.1)
erb (default: 2.2.3)
error_highlight (default: 0.3.0)
etc (default: 1.3.0)
excon (0.93.1)
faraday (1.10.2)
faraday-em_http (1.0.0)
faraday-em_synchrony (1.0.0)
faraday-excon (1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.0.4)
faraday-net_http (1.0.1)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
faraday-rack (1.0.0)
faraday-retry (1.0.3)
faraday_middleware-aws-sigv4 (0.6.1)
fcntl (default: 1.0.1)
ffi (1.15.5)
fiber-local (1.0.0)
fiddle (default: 1.1.0)
fileutils (default: 1.6.0)
find (default: 0.1.1)
fluent-config-regexp-type (1.0.0)
fluent-diagtool (1.0.1)
fluent-logger (0.9.0)
fluent-plugin-calyptia-monitoring (0.1.3)
fluent-plugin-elasticsearch (5.2.4)
fluent-plugin-filter_typecast (0.0.3)
fluent-plugin-flowcounter-simple (0.1.0)
fluent-plugin-geoip (1.3.2)
fluent-plugin-kafka (0.18.1)
fluent-plugin-metrics-cmetrics (0.1.2)
fluent-plugin-opensearch (1.0.8)
fluent-plugin-prometheus (2.0.3)
fluent-plugin-prometheus_pushgateway (0.1.0)
fluent-plugin-record-modifier (2.1.1)
fluent-plugin-rewrite-tag-filter (2.4.0)
fluent-plugin-s3 (1.7.2)
fluent-plugin-sd-dns (0.1.0)
fluent-plugin-systemd (1.0.5)
fluent-plugin-td (1.2.0)
fluent-plugin-utmpx (0.5.0)
fluent-plugin-webhdfs (1.5.0)
fluentd (1.15.3)
forwardable (default: 1.3.2)
geoip-c (0.9.1)
geoip2_c (0.3.3)
getoptlong (default: 0.1.1)
hirb (0.7.3)
http_parser.rb (0.8.0)
httpclient (2.8.3)
io-console (default: 0.5.11)
io-nonblock (default: 0.1.0)
io-wait (default: 0.2.1)
ipaddr (default: 1.2.4)
irb (default: 1.4.1)
jmespath (1.6.1)
json (2.6.2, default: 2.6.1)
linux-utmpx (0.3.0)
logger (default: 1.5.0)
ltsv (0.1.2)
matrix (0.4.2)
mini_portile2 (2.8.0)
minitest (5.15.0)
msgpack (1.6.0)
multi_json (1.15.0)
multipart-post (2.2.3)
mutex_m (default: 0.1.1)
net-ftp (0.1.3)
net-http (default: 0.2.0)
net-imap (0.2.3)
net-pop (0.1.1)
net-protocol (default: 0.1.2)
net-smtp (0.3.1)
nio4r (2.5.8)
nkf (default: 0.1.1)
observer (default: 0.1.1)
oj (3.13.17)
open-uri (default: 0.2.0)
open3 (default: 0.1.1)
opensearch-api (2.0.2)
opensearch-ruby (2.0.3)
opensearch-transport (2.0.1)
openssl (default: 3.0.0)
optparse (default: 0.2.0)
ostruct (default: 0.5.2)
parallel (1.22.1)
pathname (default: 0.2.0)
power_assert (2.0.1)
pp (default: 0.3.0)
prettyprint (default: 0.1.1)
prime (0.1.2)
prometheus-client (2.1.0)
protocol-hpack (1.4.2)
protocol-http (0.23.12)
protocol-http1 (0.14.6)
protocol-http2 (0.14.2)
pstore (default: 0.1.1)
psych (default: 4.0.3)
public_suffix (5.0.0)
racc (default: 1.6.0)
rake (13.0.6)
rbs (2.1.0)
rdkafka (0.11.1)
rdoc (default: 6.4.0)
readline (default: 0.0.3)
readline-ext (default: 0.1.4)
reline (default: 0.3.0)
resolv (default: 0.2.1)
resolv-replace (default: 0.1.0)
rexml (3.2.5)
rinda (default: 0.1.1)
rss (0.2.9)
ruby-kafka (1.5.0)
ruby-progressbar (1.11.0)
ruby2_keywords (default: 0.0.5)
rubyzip (1.3.0)
securerandom (default: 0.1.1)
serverengine (2.3.0)
set (default: 1.0.2)
shellwords (default: 0.1.0)
sigdump (0.2.4)
singleton (default: 0.1.1)
stringio (default: 3.0.1)
strptime (0.2.5)
strscan (default: 3.0.1)
syslog (default: 0.1.0)
systemd-journal (1.4.2)
td (0.16.9)
td-client (1.0.8)
td-logger (0.3.28)
tempfile (default: 0.1.2)
test-unit (3.5.3)
time (default: 0.2.0)
timeout (default: 0.2.0)
timers (4.3.5)
tmpdir (default: 0.1.2)
traces (0.7.0)
tsort (default: 0.1.0)
typeprof (0.21.2)
tzinfo (2.0.5)
tzinfo-data (1.2022.5)
un (default: 0.2.0)
uri (default: 0.11.0)
weakref (default: 0.1.1)
webhdfs (0.10.2)
webrick (1.7.0)
yajl-ruby (1.4.3)
yaml (default: 0.2.0)
zip-zip (0.3)
zlib (default: 2.1.1)