HASecuritySolutions
USB Drive Logstash Configuration error
I'm trying to use your USB drive logstash configuration that you have shown in elastic webinar I'm getting this error, Can you please give me the configuration that you used in the webinar.
PS F:\ELK\logstash-7.1.0> .\bin\logstash -f logstash1.conf --config.reload.automatic
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.runtime.encoding.EncodingService (file:/F:/ELK/logstash-7.1.0/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.Console.cs
WARNING: Please consider reporting this to the maintainers of org.jruby.runtime.encoding.EncodingService
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Sending Logstash logs to F:/ELK/logstash-7.1.0/logs which is now configured via log4j2.properties
[2019-06-03T11:18:11,781][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-06-03T11:18:11,791][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.1.0"}
[2019-06-03T11:18:14,256][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-06-03T11:18:14,397][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2019-06-03T11:18:14,438][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>7}
[2019-06-03T11:18:14,441][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>7}
[2019-06-03T11:18:14,459][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2019-06-03T11:18:14,472][INFO ][logstash.outputs.elasticsearch] Using default mapping template
[2019-06-03T11:18:14,554][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"index_patterns"=>"logstash-", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s", "number_of_shards"=>1, "index.lifecycle.name"=>"logstash-policy", "index.lifecycle.rollover_alias"=>"logstash"}, "mappings"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}
warning: thread "[main]-pipeline-manager" terminated with exception (report_on_exception is true):
SyntaxError: (ruby filter code):3: syntax error, unexpected keyword_end
eval at org/jruby/RubyKernel.java:1061
register at F:/ELK/logstash-7.1.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:59
register at org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56
register_plugins at F:/ELK/logstash-7.1.0/logstash-core/lib/logstash/java_pipeline.rb:191
each at org/jruby/RubyArray.java:1792
register_plugins at F:/ELK/logstash-7.1.0/logstash-core/lib/logstash/java_pipeline.rb:190
maybe_setup_out_plugins at F:/ELK/logstash-7.1.0/logstash-core/lib/logstash/java_pipeline.rb:446
start_workers at F:/ELK/logstash-7.1.0/logstash-core/lib/logstash/java_pipeline.rb:203
run at F:/ELK/logstash-7.1.0/logstash-core/lib/logstash/java_pipeline.rb:145
start at F:/ELK/logstash-7.1.0/logstash-core/lib/logstash/java_pipeline.rb:104
[2019-06-03T11:18:14,631][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create
warning: thread "Api Webserver" terminated with exception (report_on_exception is true):
Here is my configuration input { beats { port => 5044 } }
filter { if [event_id] == 2003 or [event_id] == 2102 { grok { match => {"[user_data][InstanceId]" => "SWD\WPDBUSENUM\??(?<usb_device>[A-Z]+)#(?<usb_typr>[A-Z]+)(&VEN_%{DATA:usb_vendor})?(&PROD_%{DATA:usb_device_product})?(&REV_%{DATA:usb_rev})?#%{INT:usb_serial}&%{INT:usb_slot}#?{%{DATA:usb_session_guid}}"} } } if [user][name] =~ "^DWM-" or [user][name] == "SYSTEM" or [user][name] == "NETWORK SERVICE" or [user][name] == "LOCAL SERVICE" or [user][name] =~ "^SVC_" { mutate { add_tag => ["service_account"]} } if [user][name] =~ /$/ { mutate {add_tag => ["machine", "noise"]} } if [user][name] != "-" { mutate {add_field => {"user_array" => "%{[user][name]}"} } } if [event_data][Payload] and [event_id] == 4103 and [source_name] == "Microsoft-Windows-Powershell" { ruby { code => "event.set('cmdlets',event.get('[event_data][Payload]').downcase.scan(/commandinvocation(([a-z0-9-]+))/)" } } translate { field => "LogonType" destination => "LogonType" dictionary => [ "2","Interactive (Console logon)", "3","Network (Connection to shared Folders)", "4","Batch (Scheduled task)", "5","Service (Service startup)", "7","Unlock (Unattended locked workstation)", "8","NetworkCleartext (logging over the network)", "8","NetworkCleartext (logging over the network)", "9","NewCredentials (run an app using RunAs Command)", "10","RemoteInteractive (used for RDP like terminal or remote assistance)", "11","CachedInteractive( Users log on using cached credentials)" ] override => true } }
output { elasticsearch { hosts => [ "localhost:9200" ] } }