beats icon indicating copy to clipboard operation
beats copied to clipboard

Add an option to report deltas during a flow period instead of cumulative stats

Open kvalliyurnatt opened this issue 1 year ago • 9 comments

Proposed commit message

Currently intermediate flows and final flows report cumulative stats for network.bytes and network.packets. Adding a config option enable_delta_flow_reports that will result in intermediate flows and final flows reporting network.bytes and network.packets for only that corresponding period instead of cumulative.

Checklist

  • [x] My code follows the style guidelines of this project
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [x] I have made corresponding changes to the documentation
  • [x] I have made corresponding change to the default configuration files
  • [x] I have added tests that prove my fix is effective or that my feature works
  • [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Use cases

Having delta stats in intermediate flows helps calculating things like total bytes for a specific time window easier without having to look at flows for other time periods to arrive at an accurate answer.

kvalliyurnatt avatar Mar 07 '24 19:03 kvalliyurnatt

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @kvalliyurnatt? 🙏. For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

mergify[bot] avatar Mar 07 '24 19:03 mergify[bot]

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 49 min 16 sec

:grey_exclamation: Flaky test report

No test was executed to be analysed.

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine avatar Mar 07 '24 20:03 elasticmachine

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

elasticmachine avatar Mar 07 '24 21:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #1591 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d
  • :green_heart: Build #1081 succeeded a329f8ae303510f8b6d9d081c9e25ae6562d000a
  • :green_heart: Build #1066 succeeded b99bf52b476aaa7d069c63575c49ae99ac6f88d7
  • :green_heart: Build #1065 succeeded 21e1beea96af39d53fa5d251b428f43822577045
  • :green_heart: Build #1064 succeeded bf209e7d5665ffca829dad780155684c857429ec
  • :green_heart: Build #1062 succeeded 0c3d245fb2ad76e97130bd134516581621ddd833

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #2428 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d
  • :green_heart: Build #1918 succeeded a329f8ae303510f8b6d9d081c9e25ae6562d000a
  • :green_heart: Build #1903 succeeded b99bf52b476aaa7d069c63575c49ae99ac6f88d7
  • :green_heart: Build #1902 succeeded 21e1beea96af39d53fa5d251b428f43822577045
  • :green_heart: Build #1901 succeeded bf209e7d5665ffca829dad780155684c857429ec
  • :green_heart: Build #1899 succeeded 0c3d245fb2ad76e97130bd134516581621ddd833

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #1156 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #3646 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d
  • :green_heart: Build #3132 succeeded a329f8ae303510f8b6d9d081c9e25ae6562d000a
  • :green_heart: Build #3117 succeeded b99bf52b476aaa7d069c63575c49ae99ac6f88d7
  • :green_heart: Build #3116 succeeded 21e1beea96af39d53fa5d251b428f43822577045
  • :green_heart: Build #3115 succeeded bf209e7d5665ffca829dad780155684c857429ec
  • :green_heart: Build #3113 succeeded 0c3d245fb2ad76e97130bd134516581621ddd833

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #1198 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #1166 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :green_heart: Build #1153 succeeded 96f1b15eac73e6d1e677dacf92cb98f184fd141d

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:broken_heart: Build Failed

Failed CI Steps

History

  • :broken_heart: Build #3556 failed a329f8ae303510f8b6d9d081c9e25ae6562d000a
  • :broken_heart: Build #3541 failed b99bf52b476aaa7d069c63575c49ae99ac6f88d7
  • :broken_heart: Build #3540 failed 21e1beea96af39d53fa5d251b428f43822577045
  • :broken_heart: Build #3539 failed bf209e7d5665ffca829dad780155684c857429ec

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :broken_heart: Build #2215 failed a329f8ae303510f8b6d9d081c9e25ae6562d000a
  • :broken_heart: Build #2200 failed b99bf52b476aaa7d069c63575c49ae99ac6f88d7
  • :broken_heart: Build #2199 failed 21e1beea96af39d53fa5d251b428f43822577045
  • :broken_heart: Build #2198 failed bf209e7d5665ffca829dad780155684c857429ec
  • :broken_heart: Build #2196 failed 0c3d245fb2ad76e97130bd134516581621ddd833

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

:green_heart: Build Succeeded

History

  • :broken_heart: Build #1925 failed a329f8ae303510f8b6d9d081c9e25ae6562d000a
  • :green_heart: Build #1910 succeeded b99bf52b476aaa7d069c63575c49ae99ac6f88d7
  • :green_heart: Build #1906 succeeded 0c3d245fb2ad76e97130bd134516581621ddd833

cc @kvalliyurnatt

elasticmachine avatar Mar 20 '24 13:03 elasticmachine

@haesbaert any idea why the filebeat tests are failing, they don't seem to be related to my changes .

kvalliyurnatt avatar Mar 20 '24 22:03 kvalliyurnatt

@haesbaert any idea why the filebeat tests are failing, they don't seem to be related to my changes .

No idea, I'll ping some people, I don't think it should block this PR though.

haesbaert avatar Mar 21 '24 10:03 haesbaert