[Heartbeat] Fix expiration of cert chains calculation

[andrewvc]

Fixes https://github.com/elastic/beats/issues/33215

This is a bit WIP in that I'm not certain this is the final logic we want. It appears the elastic agent libs override the default golang TLS implementation's verification and only set the cert chains attribute on the connection when strict validation is enabled, where the default is full. See https://www.elastic.co/guide/en/beats/heartbeat/current/configuration-ssl.html for definition of the various validation options.

I suspect we'd be best off making strict the default since this is what chrome does nowadays, requiring a SAN to be set, which is the only distinction between the checks of strict and full.

To test this use the following config, which returns a too-early expiration date without this patch:

- type: http
  enabled: true
  id: wyportal
  name: WYPortal
  schedule: "@every 1m"
  ssl.verification_mode: strict
  urls: "https://portals.edu.wyoming.gov/"

You can try playing with the verification mode as well

What does this PR do?

Why is it important?

Checklist

• [x] My code follows the style guidelines of this project
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] I have made corresponding change to the default configuration files
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

Use the above config and run and build with the following command to prune the output. Note this requires your heartbeat yaml to not have an output section set.

mage build && ./heartbeat -e  -E output.console={} | jq '{"id": .monitor.id, "na": .tls.server.x509.not_after}'

[elasticmachine]

Pinging @elastic/uptime (Team:Uptime)

[mergify[bot]]

This pull request does not have a backport label. If this is a bug or security fix, could you label this PR @andrewvc? 🙏. For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

:green_heart: Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-10-11T14:15:49.652+0000

• Duration: 46 min 44 sec

Test stats :test_tube:

Test	Results
Failed	0
Passed	1869
Skipped	25
Total	1894

:green_heart: Flaky test report

Tests succeeded.

:robot: GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify[bot]]

This pull request is now in conflicts. Could you fix it? 🙏 To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-cert-chains upstream/fix-cert-chains
git merge upstream/main
git push upstream fix-cert-chains

[andrewvc]

@emilioalvap this is now ready for review again, I spent probably a bit too much effort improving the logic and test cases in aa35bb0 , while I think it could be cleaned up further, given that this requires strict, I think it covers more than can probably be justified given the effort of digging up the right certs.

[paulb-elastic]

Taking the PR off the board as the parent issue is on there

[andrewvc]

@Mergifyio backport 8.5

[mergify[bot]]

backport 8.5

✅ Backports have been created

• #33316 [Heartbeat] Fix expiration of cert chains calculation (backport #33231) has been created for branch 8.5

[andrewvc]

@Mergifyio backport 7.17

[mergify[bot]]

backport 7.17

✅ Backports have been created

• #33317 [Heartbeat] Fix expiration of cert chains calculation (backport #33231) has been created for branch 7.17