Disable logging of conntrack stats traffic

[shawniverson]

Description

Fixes: #6718

Changes the default behavior of conntrackd to stop logging verbose conntrack-ed traffic to /var/log/daemon.log. This is expensive in terms of performance and also costly in terms of the limited disk capacity in a VR. In cases of large quantities of traffic, /var fills up and causes the VR to either stop passing traffic or perform necessary tasks in the VPC.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [ ] Minor

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [x] Major
• [ ] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Change applied to opt/cloud/templates/conntrackd.conf.templ in systemvm.iso inside of cloud-scripts.tgz. systemvm.iso distributed to all Cloudstack KVM hosts and placed at /usr/share/cloudstack-common/vms/systemvm.iso. Redundant VPC restarted with cleanup. Confirmed change present and monitored /var/log/daemon.log for traffic entries while generating traffic via a private gateway using iperf.

[boring-cyborg[bot]]

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) Here are some useful points:

• In case of a new feature add useful documentation (raise doc PR at https://github.com/apache/cloudstack-documentation)
• Be patient and persistent. It might take some time to get a review or get the final approval from the committers.
• Pay attention to the quality of your code, ensure tests are passing and your PR doesn't have conflicts.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Issues, Mailing list and Slack.
• Be sure to read the CloudStack Coding Conventions. Apache CloudStack is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: [email protected] (https://cloudstack.apache.org/mailing-lists.html) Slack: https://apachecloudstack.slack.com/

[codecov[bot]]

Codecov Report

Merging #6738 (4454b2e) into 4.18 (d958882) will increase coverage by 0.00%. Report is 3 commits behind head on 4.18. The diff coverage is 75.00%.

@@            Coverage Diff            @@
##               4.18    #6738   +/-   ##
=========================================
  Coverage     13.02%   13.02%           
- Complexity     9029     9030    +1     
=========================================
  Files          2720     2720           
  Lines        256998   257001    +3     
  Branches      40081    40083    +2     
=========================================
+ Hits          33462    33463    +1     
- Misses       219335   219337    +2     
  Partials       4201     4201           

Files Changed	Coverage Δ
...ervisor/kvm/resource/LibvirtComputingResource.java	18.31% <60.00%> (+0.01%)	:arrow_up:
...om/cloud/hypervisor/kvm/resource/LibvirtVMDef.java	67.13% <100.00%> (ø)
...er/src/main/java/com/cloud/hypervisor/KVMGuru.java	50.81% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4190

[rohityadavcloud]

@blueorangutan test

[blueorangutan]

@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 4378

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4397

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with SystemVM template(s). I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_multiplication_x: suse15. SL-JID 4403

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 4404

[rohityadavcloud]

@blueorangutan test

[blueorangutan]

@rohityadavcloud a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian test result (tid-5093) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 45525 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6738-t5093-kvm-centos7.zip Smoke tests completed. 104 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[DaanHoogland]

@shawniverson I'm marking this PR as draft for now

[DaanHoogland]

@shawniverson can you resolve the conflicts here, please?

[shawniverson]

@DaanHoogland I need guidance on the source code. I need to add a global setting for this to toggle the logging in conntrackd, but I am struggling to find where in the source this change would be made. Sorry, but I'm not terribly familiar with the source and it is quite vast.

[DaanHoogland]

ok @shawniverson , that is a big question. If you need more work than this done or are maintaining a cloud it can pay off to work through the hackerbook. The making of a global setting is done by defining a ConfigKey<> in a service. I think in your case it should be the NetworkService but there could be a better location. Next the value of the setting must be passed to the actual virtual router. Finally the code that installs the template that you editted must take the value of the setting into account.

I understand that this can be daunting especially as you say, because the code base is huge. Please ask any questions you have. I may know the answer and if not, I may know someone that does.

[shawniverson]

@DaanHoogland Thank you that helps a lot. This looks like a good starting point for me to work on this. I can track some of the existing settings defined here through the source to application on the VRs themselves, I believe.

https://github.com/apache/cloudstack/tree/main/server/src/main/java/com/cloud/network/router

[DaanHoogland]

yes, VirtualNetworkApplianceManager seems like a good start. Good luck and let us know ;)

[DaanHoogland]

@shawniverson are you still working on this?

[shawniverson]

Well, I'm trying to figure out how to get the global setting into the vr configuration scripts. I'm still working on this.