passivedns icon indicating copy to clipboard operation
passivedns copied to clipboard

Plugin output

Open lafeuil opened this issue 9 years ago • 6 comments

I would like to have your opinion on a new feature. Do you think you could agree to add a plugin system to handle output of passivedns ?

There is a new similar system in dnscap. In command line, you can pass plugins (.so) with -P argument. A plugin is a dynamic library. Each plugin can implement functions :

void <plugin_name>_usage();
void <plugin_name>_getopt(int *argc, char **argv[]);
void <plugin_name>_start();
void <plugin_name>_stop();

and must implement an output function :

void <plugin_name>_output(pdns_record *l, pdns_asset *p, ldns_rr *rr, ldns_rdf *lname, uint16_t rcode);

With this system, we can implement some outputs like a massive inject in a database with better performance than parsing log file.

Do you think you can integrate a patch with this feature ?

lafeuil avatar Jan 05 '16 09:01 lafeuil

If this is a feature you want, then you could write a patch, and send us a pull request :)

There is a million things I want to add to both passivedns and other projects, but time is a limiting factor.

thus avatar Jan 05 '16 14:01 thus

I have created a first version of a plugin system to handle output of passivedns. You can see it in my repository : https://github.com/lafeuil/passivedns You can see the change in this commit : lafeuil/passivedns@fc2ccde5a614a3ddf6d3a2f07c255a904339901f

The commit have an example with a JSON plugin output. It is a copy of the current output JSON implementation. You can build it with :

cd src/plugins/json
libtoolize && aclocal && autoconf && automake && ./configure && make

You can run passivedns with the -O parameter :

./src/passivedns -L - -l - -r dns-exemple.pcap -O
src/plugins/json/.libs/json.so -L - -l -

Tell me what do you thing about my patch.

I think the plugin output is a good idea when I see the new pull request #70 about Redis.

lafeuil avatar Jan 11 '16 13:01 lafeuil

Think this would be a good idea aswell, i felt that the code (when introducing Redis) became abit cluttered having to take into account the differet ifdef's and so on. A plugin system would make it easier to integrate new modules in a clean fashion perhaps?

But then again, i see that the proposed JSON output plugin logs to file, and the the Redis output would have to be implemented both here, and in the regular output format?

maxtors avatar Jan 11 '16 14:01 maxtors

@lafeuil: Nice! I haven't gotten around to take a proper look at the code yet, but I really like the concept :)

@Maxtors: I don't see why the Redis output couldn't be implemented like this. Could you explain why?

thus avatar Jan 11 '16 15:01 thus

@Maxtors My implementation doesn't support more than one plugin. But If you want, I can change this and you can chain output plugins :

./src/passivedns -L - -l - -r dns-exemple.pcap -O src/plugins/txt/.libs/txt.so -L - -l - src/plugins/json/.libs/redis.so -h redis-host.com

The default output implementation can be implemented to a "txt" plugin and you can implement a "Redis" plugin.

Another solution is to provide two mode when we define a plugin in the command line :

  • -o without default output
  • -O with default output

If we want integrate this plugin feature, we must define how it work exactly. I think the json output must be implemented to a plugin. But how the default output must be implemented ? In a plugin ? in the passivedns code ?

lafeuil avatar Jan 11 '16 15:01 lafeuil

My thought was that, the redis output (or say a SQL, Cassandra, Syslog, Kafka, what not) is more of a "where to output" not a "the format of the output", so the styling (formatters) like JSON or "default" should be seperated into perhaps "formatters" plugins?

Say if someone in the future wants to create a CEF formatter, or maybe a BSON output, but wants to use syslog on one server, but redis on the others.

maxtors avatar Jan 11 '16 16:01 maxtors