kunai-project
[featurerequest] send events logs to remote syslog/journald/http...
It would be great if kunai had the option to send logs to a remote system, be it through classical syslog, journald, http or else. Benefits
- logs consolidation
- avoid to write any local disk and possibly alter evidence during investigation Thanks! Great work!
Hey @juju4,
This is my ToDo since a while but I kept postponing it as nobody asked for it ... Now you did it might be the good timing to implement it.
My idea was to implement a minimal kunai server talking websocket with agents to:
- send config/rules
- receive logs
Would that be acceptable for you or you'd need something more standard such as syslog ?
Other question: if you'd have this would you still need feature #177 ?
On management side, that was definitely a coming question :) On logs, I would prefer to stay agnostic and allow a central management server and/or a logging/SIEM system. Most of them can receive logs through HTTP but not websocket AFAIK. Examples:
- https://www.elastic.co/guide/en/integrations/current/http_endpoint.html
- https://openobserve.ai/docs/api/ingestion/logs/json/
- https://docs.cribl.io/stream/sources-cribl-http/
- https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/
If I take osquery case, there is a config and a logging plugin with an endpoint for each (--config_plugin/config_tls_endpoint/config_tls_refresh and --logger_plugin/logger_tls_endpoint/logger_tls_period). It's also possible to have multiple logger output like local and remote. Management server can be the one getting logs or not.
As for #177, yes, this one is a good workaround.
My idea was more implementing a light kunai server to communicate with agents. I tend to think it is very risky to expose log storage platforms to every agent (potentially host) in the network. Maybe I am too much paranoid ...
So my idea was more something like the following:
graph LR;
A[Kunai Agent] -->|Websocket| B[Kunai Server];
B -->|Forward Logs| C[OpenSearch];
B -->|Forward Logs| D[Elastic];
B -->|Forward Logs| E[Splunk];
B -->|Forward Logs| F[...];
B -->|Websocket| A;
If that is something that would fit your needs I can prioritize implementation
In the full paranoid mode, I like kunai not doing any networking and leave logs shipping to another tool :) Behavior seems more like a proxy and a single point of failure if not supporting multi-targets. If I look to other tools
- Commercial all have their own central management server doing logging
- Osquery have multiple management servers (fleet, zentral...) that can manage logs or not
- Velociraptor have its management server but can ship artefacts to a variety of targets like amazon s3 or azure storage
- In my xp, logging collector are set per environment and exposed to those, like often web proxy, dns server, vulnerability scanner, monitoring server and else. Adding another one won't necessarily change much but is also another server to manage. In the end, that's your call.
Ok, so if I understand correctly, if I resolve #177 it would actually solve your problem with log collection.
Another option which hasn't been mentioned, what about setting kunai output to stdout so that the logs ends-up in journald ? I don't have the same knowledge as you in log collection but wouldn't it be an easy way to forward logs to a remote server thanks to whatever tool supporting journald uploads ?
Yes, good if using local journald (or syslog) do remote shipping too and likely the easier one to implement.
@juju4, this is already possible to print log in stdout, just set /dev/stdout instead of a file in configuration file (see: https://why.kunai.rocks/docs/configuration).
# Output configuration
output:
# Where the events should be written.
path: /dev/stdout
Maybe what would need to be done is a config parameter to also dump the kunai (error, warning, info ...) into json so that the service output only contains json formatted output and not a mix of text and json.
Yes, that works that way (requires buffered false too). For now as I filter on parseable json for my upstream log collection, no immediate issue but yes, better to get service errors and the like.
Hey @juju4,
Do you think you could share the architecture of your setup somewhere ? To help others with their integration :).
sure. need to clean stuff. short version is: kunai > opentelemetry-collector-contrib > openobserve
@juju4
If you feel OK with sharing your kunai experience, you can do it here: https://github.com/kunai-project/community-hub I believe it would be super useful to have your inputs there :)