Refactor FlowAggregator exporter code

[antoninbas]

trafficstars

We currently support 2 different exporters for the FlowAggregator: the IPFIX exporter and the ClickHouse exporter. As I plan to add a third one, which will upload flow record to AWS S3, it is good to unify exporter implementations behind a single interface.

Before this change the IPFIX exporter was deeply embedded within the main FlowAggregator logic. With this change, it is treated as any other exporter.

This change also tries to improve thread-safety (in particular for live updates of exporter configuration options) and adds a few unit tests.

Signed-off-by: Antonin Bas [email protected]

[codecov[bot]]

Codecov Report

Merging #4080 (3217aac) into main (576b080) will increase coverage by 0.61%. The diff coverage is 69.97%.

@@            Coverage Diff             @@
##             main    #4080      +/-   ##
==========================================
+ Coverage   66.93%   67.54%   +0.61%     
==========================================
  Files         298      300       +2     
  Lines       45465    45539      +74     
==========================================
+ Hits        30431    30759     +328     
+ Misses      12635    12383     -252     
+ Partials     2399     2397       -2     

Flag	Coverage Δ
e2e-tests	40.41% <ø> (?)
integration-tests	35.37% <ø> (+0.03%)	:arrow_up:
kind-e2e-tests	50.34% <57.88%> (+1.04%)	:arrow_up:
unit-tests	44.47% <42.76%> (+0.18%)	:arrow_up:

Impacted Files	Coverage Δ
pkg/flowaggregator/exporter/clickhouse.go	54.34% <54.34%> (ø)
pkg/flowaggregator/exporter/ipfix.go	63.74% <63.74%> (ø)
...lowaggregator/clickhouseclient/clickhouseclient.go	82.71% <77.46%> (-0.66%)	:arrow_down:
pkg/flowaggregator/flowaggregator.go	69.44% <91.66%> (+11.27%)	:arrow_up:
pkg/ipfix/ipfix_process.go	81.25% <0.00%> (-18.75%)	:arrow_down:
pkg/util/k8s/name.go	8.00% <0.00%> (-16.00%)	:arrow_down:
pkg/controller/externalippool/controller.go	83.03% <0.00%> (-8.04%)	:arrow_down:
pkg/agent/flowexporter/utils.go	76.59% <0.00%> (-4.26%)	:arrow_down:
pkg/controller/networkpolicy/status_controller.go	65.10% <0.00%> (-3.24%)	:arrow_down:
pkg/agent/openflow/multicluster.go	44.85% <0.00%> (-1.87%)	:arrow_down:
... and 32 more

[antoninbas]

@wsquan171 thanks for the review. I addressed your comments. I also had to make some changes (see last commit) because the new implementation of IPFIX exporter had a bug that was revealed by the e2e tests. I also added some more unit tests.

[antoninbas]

/test-all

[antoninbas]

@heanlan @yanjunz97 Thanks for the reviews, I addressed your comments

[antoninbas]

/test-all