docs(proxy): add ssl guidelines for communication between Agent and Datadog when using a proxy

[Kaderinho]

What does this PR do?

• Add guidelines on raw/encrypted communication between the Agent, proxy and Datadog intake.
  • Add Datadog certificate verification on the proxy host.
  • Add docs on when and how to encrypt communication between the Agent and the proxy
• Fix some errors :
  • telemetry HAProxy frontend was missing after a merge
  • AppSec port was different on HAProxy and Nginx + missing in one datadog.yaml config
  • update the part about skip_ssl_validation (which was previously useless because HTTP was used between the Agent and the proxy)

Motivation

Give better guidelines to customers

https://docs-staging.datadoghq.com/nicolas.guerguadj/aml-100-add-https-guideline-for-proxy-setup/agent/proxy/?tab=agentv6v7

Additional Notes

• Did not test network_devices but as it is using the same component as database_monitoring, the behavior is the same
• There is still no guidelines on how to create certificates, will be addressed in a future PR

To check

There was some modifications on this file before this PR that introduced some errors like missing frontend or inconsistencies with ports. I realized it and fixed them after all I added, so I might have missed some stuff :

It would be nice if someone check after me that :

• [ ] HTTP configuration (the one by default before this PR) should only have :
  • [ ] ssl verify <PATH_TO_CERTIFICATES> instead of ssl verify none for HAProxy
  • [ ] proxy_ssl_verify on added to all server for NGINX
• [ ] HTTPS configuration should contains the HTTP config information and :
  • [ ] ssl crt <PATH_TO_PROXY_CERTIFICATE> at the end on each bind line for HAProxy
  • [ ] ssl at the end on each listen line for NGINX
• [ ] There is no HAProxy reference in NGINX part and vice versa

---

Reviewer checklist

• [ ] Review the changed files.
• [ ] Review the URLs listed in the Preview section.
• [ ] Check images for PII
• [ ] Review any mentions of "Contact Datadog support" for internal support documentation.

[github-actions[bot]]

Preview links (active after the build_preview check completes)

Modified Files

• https://docs-staging.datadoghq.com/nicolas.guerguadj/aml-100-add-https-guideline-for-proxy-setup/agent/proxy

[maycmlee]

Created DOCS-3856 for editorial review.

[sgnn7]

Hey @Kaderinho, I briefly mentioned the new ssl verify <path> guidance to @olivielpeau today as it impacts the FIPS proxy deployments as well but we didn't have enough time to resolve the path forward. We have a meeting tomorrow (Wed, Aug 3) on figuring out how we want to handle the ssl verify ... and I'll do a follow-up review after that. Sorry about the delay!

[Kaderinho]

Hi @hestonhoffman,

Sorry for the delay 🙇 I have applied the suggestions (04e253865ed771231eb07331eed5003768a9764d) and fix an error in the HAProxy config (3919cfe80e4b9a481d8883649ecb1fa5663c1230).

I'll do a final check tomorrow morning to verify one last time that both configuration are correct (just in case) but if everything is good for you, please feel free to approve and I'll merge it tomorrow 🙏

[Kaderinho]

I applied suggestions and correct one file (missing ;), I checked HTTP and HTTPS configuration for both HAProxy and NGINX and that LGTM so it's ready to merge on my side

Thanks for the review !