Improve "Deploy" documentation to provide clear best practices

[lukeheath]

Goal

User story
As a potential Fleet user,
I want to understand the recommend best practice for deploying Fleet
so that I can quickly and easily try Fleet.

Our north star: https://fleetdm.com/handbook/company/why-this-way#why-read-documentation

Changes

Customer Success

@ksatter is available to assist with small tasks but not to own deliverables for this story. Please keep her in the loop for discussions so she has context for customer conversations.

• [ ] @nonpunctual Outdated documentation changes: Update "Deploy" guide:

1. Add a simple, easy-to-read, recommended best practice deployment guide for all Fleet instances.
2. Be more opinionated. Tell readers the best practice for their host count. Be confident in our best practices.

• Up to 1,000 hosts
• Up to 25,000 hosts
• Up to 150,000 hosts
• Up to 300,000 hosts

3. Separate all deployment reference docs so they are clearly separate from the guide. Deployment reference docs are for power users who want to customize Fleet, not for new users.

Engineering

• [ ] TODO @rfairburn Provide Terraform best practice examples that are plug-and-play for each recommended host tier.

• Be opinionated. Make it as easy and simple as possible.
• For example, include carve and log destinations configured via Terraform. That way the new user doesn't have to think about it.

• [ ] TODO @lukeheath ritual for making sure current versions are tagged in a way that's compatible with the published docs

• Make sure to update the "Releasing Fleet" docs and related scripts once Robert has examples in place that we want to keep up-to-date.

Context

• Requestor(s): @lukeheath

[ksatter]

Current AWS Terraform deployment guide:

https://fleetdm.com/docs/deploy/deploy-on-aws-with-terraform

[lukeheath]

@rfairburn I won't have time to dog into this TODO so I've re-assigned to you:

• [ ] TODO @rfairburn: What effort would be involved in separating out all jobs to the dedicated runner?

• Could a single instance handle all jobs concurrently? Up to what point?

[lukeheath]

@rfairburn I think it was originally assigned to me because we thought it might need engineering effort. But I'm wondering if we could just use the same approach as Ben implemented for the vuln scans in a separate worker process? Could we just move all jobs over there?

[rfairburn]

@rfairburn I think it was originally assigned to me because we thought it might need engineering effort. But I'm wondering if we could just use the same approach as Ben implemented for the vuln scans in a separate worker process? Could we just move all jobs over there?

@lukeheath Ben's approach took code in Fleet and configuration flags to reference said code on each container in order for this to work.

if we want this method to be used for all crowns, then we would definitely still need engineering backend work whether it was from one of us or ticketed and assigned to the wider engineering team.

[lukeheath]

@rfairburn Got it, thanks for the clarification. I'm taking this one back and I'll bring to the engineering team to review.

[pacamaster]

https://github.com/fleetdm/fleet/pull/17355 - updated "example" fleet.tf and the readme. This is "barebones" to get Fleet stood up. The readme was also updated to reflect commands to run. Will need to update the actual doc. Have confirmed able to get Fleet going with this example terraform.

Some points or notes

• The guide already is as "simple" as we can get for terraform, do not see any other way to make it anymore stripped down
• The actual guide is also taken from an old doc, so some examples and infra explanations might be below the fold
• We've stripped out modules for SES, firehose logging, etc as think this is added complexity and out of the scope for this documentation

[lukeheath]

From the check-in call today:

• This effort aims to show a clear and straightforward path for deploying Fleet in enterprises at scale.
• We're not trying to provide a quick and easy way to try Fleet for the first time; we're demonstrating how easy it can be to deploy at scale.
• The introduction page shows a single, best-practice approach to deploying Fleet at scale. (Terraform on AWS)
• The reference documentation includes more comprehensive information about other deployment scenarios and providers.

[lukeheath]

@nonpunctual Since the sprint wrapped up on Friday, I wanted to check in and see what the ETA is for a PR to the deploy page.

[nonpunctual]

@lukeheath Met with Kathy & Grant on Fri to discuss their ideas for what should be on the page. Tried to have a chat with Rachael on Fri about final design. I guess I will just have to submit PR without that. WIll put it in today & Robert says his docs can be linked to it today or tomorrow.

[nonpunctual]

https://github.com/fleetdm/confidential/issues/5358 Issue from @ambrusps on making prospect training & customer onboarding smoother.

[nonpunctual]

• Met with @lukeheath today who OK'd the overall concept for reorganizing web site pages related to "Deploy Fleet".
• @mike-j-thomas has the mock-up & @nonpunctual will meet with him again to go over design details & linking.

Open questions:

• How will we present options for different platforms with OS picker?
  • macOS + Linux are same?
  • Windows has prereq for WSL?
• Where on the page will the Ref Arch / scale stuff be?
  • Higher?
• Do we need any additional PR from @rfairburn for this to work? (If so, I don't think it's as urgent as this 1st pass...)
• @rfairburn @edwardsb What else do we need to say about configuring DNS in the prereq section?

See: "Fleet Server Setup Best Practices & Docs" Google Doc for mock-up.

[mike-j-thomas]

Hey @nonpunctual, here's a link to the Figma draft for /deploy. Let's fill in the blanks and get it to design review early next week. I left a todo for you (in red text in Figma) to curate the existing list of guides in order to lower the barrier of entry.

https://www.figma.com/file/3he8e72251IEnF6dBafKq1/%F0%9F%9A%A7-fleetdm.com-(scratchpad)?type=design&node-id=11878-123264&mode=design

[nonpunctual]

Thanks @mike-j-thomas I was dreaming about it last night. :) There are a few steps in instructions we'll need to add but I am still pretty happy with the overall. 1 thing to consider I think: once we have all the necessary info, how can we make it look more like fleetctl-preview page, generally? No action, just simplify, simplify simplify...

@lukeheath if you get a chance please take a look at the "real" mock up linked above. Thanks!

[lukeheath]

@nonpunctual Lookin' good! If you let me know when a design review is for this I'll try to attend.

[lukeheath]

@nonpunctual I wanted to check-in on the status of this story. Is there anything I can do to help?

[nonpunctual]

Hi Luke - good timing. :) I set up a meeting Monday to close out on this with Mike Thomas tonight but he rescheduled for tomorrow. I think this should be the last step & then it can go to design review for publishing.

[nonpunctual]

@lukeheath Design review scheduled for 20240415 with Mike McNeil, Eric, Mike T (tenative), me & you (optional). Thanks.

[nonpunctual]

Phase 2 notes:

Terraform code can be tagged to specific versions. This page should ultimately display those versions & automatically update them

AWS Marketplace

Each advanced guide could have its own Upgrade Fleet section.

[nonpunctual]

Met with @mike-j-thomas to create final layout design & enter instruction steps.

Need edited video of terraform install.

https://docs.google.com/document/d/1ZBqWISpPo3mg3_0uFlfTtvqnrO6cyJnzy6EV3PwqrzM/edit

[lukeheath]

@mike-j-thomas @nonpunctual @eashaw Hey y'all!

I've updated the ticket to reflect the current status. There's a new "Digital Experience" section at the bottom with the final steps to get this live for @mike-j-thomas (designs may have been done already) and @eashaw to implement the change.

I also updated the assignees to include folks who will be needed to complete and close this story today.

[nonpunctual]

I will keep my eye on this & we will immediately put in PRs to clean up any of the instruction steps & add a video run-through for AWS.

[eashaw]

I saw that the designs for this have been marked ready for dev in Figma and I made https://github.com/fleetdm/fleet/pull/18992 to add the new page.

@nonpunctual Can you help me get the URLs I need for links in the AWS section of the guide?

1. https://github.com/fleetdm/fleet/blob/d5200ed72645b2c14ae142c28999f30fe60c0b9d/docs/Deploy/deploy-fleet.md?plain=1#L90
2. https://github.com/fleetdm/fleet/blob/d5200ed72645b2c14ae142c28999f30fe60c0b9d/docs/Deploy/deploy-fleet.md?plain=1#L92

[nonpunctual]

Hi @eashaw the current URL for the advanced terrafor guide = https://fleetdm.com/docs/deploy/deploy-on-aws-with-terraform

Amazon drives me crazy.

I wanted to direct them here: https://signin.aws.amazon.com/signin

That gives you a 400 because they want to send you to a region like USeast. So, I guess we need to direct them here:

https://aws.amazon.com/iam/

& have an instruction to click the "Get Started With iAM" button which will redirect to the user's current region.

[nonpunctual]

related: https://github.com/fleetdm/fleet/pull/19207

[nonpunctual]

Followed up with @mike-j-thomas @dherder @rfairburn

Final revisions of page design up to Mike T.

[fleet-release]

Clear docs guide users, Fleet deployment made simple, Peace in cloud city.