netsampler
FlowDirection is commented out
Currently the FlowDirection field is missing in the proto file because it was commented out https://github.com/netsampler/goflow2/blob/99f3629787b90c2e7241742c2b8c138d392780d4/pb/flow.proto#L20
Hi @fionera Thank you for raising the issue.
In an effort to reduce some fields that were only available in NetFlow/IPFIX, I ended up making flow direction configurable via the mapping file.
https://github.com/netsampler/goflow2/blob/99f3629787b90c2e7241742c2b8c138d392780d4/cmd/goflow2/mapping.yaml#L30-L33
The following should automatically add the direction
goflow2 -mapping mapping.yaml
Please note that v2.2.0 made some changes to the producer libraries to support custom parsing more programmatically.
Don't hesitate if you have more questions
I am using goflow as a library including the protofile. Is there any documentation on how to use this new way?
Unfortunately, I didn't get around to write proper documentation.
But in short:
You need to use a TemplateMapper implementation.
To be passed to ConvertNetFlowDataSet function that decodes NetFlow/IPFIX.
Or if you want to call CreateProtoProducer, you need to use ProtoProducerConfig which wraps a TemplateMapper.
The CLI/app does a YAML configuration conversion to a structure with the Compile function but the recent changes were to make this more abstracted.
In my case I create the flow proto struct manually https://github.com/monogon-dev/NetMeta/blob/main/cmd/portmirror/iface.go#L135. So I guess CreateProtoProducer would be the way to go?
@fionera looking at your code I believe you have the following option: updating the library and its protobuf. You would have to use protreflect's functions to populate such field (for interface capture). This may affect if you're printing the data from the structure but binary serialization would be ok.
For the various flow decoding, CreateProtoProducer would work yes.
I was pondering making the protobuf structure more abstract (eg: must have getters/setters for the producer to use) but the amount of work was quite consequent due to the lack of generics.
Totally forgot this, sorry ^^' Without the fields being in the Protobuf description, how would one use them in smth like Clickhouse?