Support third party file/stream writing libraries

[ido]

libpcap currently uses fread/fwrite directly when reading/writing from files (via pcap_dumper).

Instead of making pcap_dumper_t a FILE* directly, and having pcap_dump_open/_close/pcap_dump call fopen/fread/fwrite/fclose, please abstract away pcap_dumper_t with function pointers in a way similar to the way pcap_t works. These function pointers can then be exposed in pcap-int.h similar to how pcap_t read ops pointers are exposed.

This would allow advanced users to override pcap_dump etc. to write to compressed files, network streams, compressed network streams, etc.

One use case for this is to make it possible for the user to support streaming/fast compression formats like snappy, lz4, etc. compressed pcap files directly (without an intermediate uncompressed file), and make pcap_dump write compressed files directly (or, for example, stream those compressed dumps over a TCP socket).

Currently, this is only possible to do by forking libpcap, making the pcap_dumper_t more indirect than a simple FILE* (and exposing the function pointers in pcap-int.h) would make it possible for users who want to write compressed pcap files directly to be able to use mainline libpcap rather than fork it, similar to how they are able to do so currently for reading pcap files (with pcap_t->read_op)...

[guyharris]

Reading, or writing? You're talking about writing files in the body, but reading them in the title.

For writing a file, the one risk is that code that assumes that a pcap_dumper_t * is a FILE *; programs shouldn't do that if they don't have to, and, while, for example, tcpdump does so if libpcap doesn't have pcap_dump_ftell(), libpcap with pcap_dumper_t * not being a FILE * will have pcap_dump_ftell(), as well as pcap_dump_flush(), so there probably shouldn't be a need to directly access the FILE *.

[ido]

Sorry about that, fixed the title. I'm talking about writing files and/or streams. Reading is currently sufficiently indirect/abstract that I can replace the read / get next packet operation with one that, e.g., gets it from a database, stream, or compressed file. I would like to also be able to do that with writing, which means making pcap_dumper_t more abstract/indirect than simply a FILE*, since I don't want to have to preload/override fwrite/ftell/etc.

[ido]

The intention would be to abstract away from libpcap the actual file being written to, so the user could make it a file, socket, compression library stream/file, encryption library stream/file object, etc.

The idea is that there is no FILE* to begin with in at the libpcap level, just buffers provided by the underlying write ops/library/plugin that libpcap is using to write the file. So, the internal procedure for using such an "output plugin" would probably look like void* pcap_dump_alloc_buffer(len) (returns ptr to buffer of size len allocated by output plugin), write pcap packet to the buffer, bool pcap_dump_commit_buffer(pd_t, buf, len) (returns true = buffer ownership transferred, false = buffer ownership returned to caller)...or similar, instead of just fwrite'ing directly.