Install fleetd with custom configuration during new Mac setup experience

[noahtalerman]

Goal

User story
As an IT admin who's asked by my security team to point fleetd to a URL that's different than Fleet server URL,
I want to customize fleetd config during Mac setup experience (ADE)
so that I can enroll my hosts to Fleet.

Key result

Deliver prioritized customer requests

Original requests

• #24305

Context

• Product designer: @marko-lisica

Changes

Product

• [ ] UI changes: Figma link
• [ ] CLI (fleetctl) usage changes: No changes
• [ ] YAML changes: #25606
• [ ] REST API changes: #25606
• [ ] Fleet's agent (fleetd) changes: No changes
• [ ] Activity changes: No changes
• [ ] Permissions changes: No changes. Already covered by "Edit macOS setup experience" row in the permissions guide.
• [ ] Changes to paid features or tiers: Available in Fleet premium only
• [ ] Transparency changes: No changes
• [x] First draft of test plan added
• [ ] Other reference documentation changes: No changes
• [ ] Once shipped, requester has been notified
• [ ] Once shipped, dogfooding issue has been filed

Engineering

• [x] Test plan is finalized
• [ ] Feature guide changes: #26074
• [ ] Database schema migrations: None
• [x] Load testing: None

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Requires load testing: no
• Risk level: Low / High high
• Risk description: This adds existing functionality to our ADE workflow which is a core feature to Fleet. It's important no regression occurs when macOS hosts are enrolled and that the custom workflow works as expected for the customer.

Test plan

• [ ] On Controls > Setup experience > Bootstrap package the option to "Install Fleet's agent (fleetd) manually" is greyed out if no bootstrap package has been uploaded.
• [ ] On Controls > Setup experience > Bootstrap package the option to "Install Fleet's agent (fleetd) manually" is selectable if a bootstrap package is uploaded.
• [ ] IF a bootstrap package is uploaded, when the user selects "Install Fleet's agent (fleetd) manually" in the UI on Controls > Setup experience > Bootstrap package (under advanced options) then fleetd IS NOT installed as part of ADE flow.
• [ ] IF a bootstrap package is uploaded, when the user does not select the "Install Fleet's agent (fleetd) manually" in the UI on Controls > Setup experience > Bootstrap package (under advanced options) then fleetd IS installed as part of ADE flow.
• [ ] IF a bootstrap package is removed when the "Install Fleet's agent (fleetd) manually" option is selected, the option should be deselected, and fleetd IS installed as part of ADE flow.
• [ ] Make sure that copy is updated in a confirmation modal after the user clicks delete bootstrap package button in the UI.
• [ ] Make sure that the user can set the option to install fleetd manually via API and that it's not installed during ADE flow. (In "Upload bootstrap package" endpoint package is required field, so user must add it before enabling manual fleetd install.)
• [ ] Make sure that the user can set the option to install fleetd manually via GitOps (YAML files) and that it's not installed during ADE flow.
• [ ] GitOps: Confirm that is user presented with error message if user tries to enable manual_agent_install without bootstrap_package specified.
• [ ] Make sure that there's no border around table that shows install status of bootstrap package on Controls > Setup experience > Bootstrap package page.
• [ ] Make sure that the order of menu items on Controls > Setup experience page is as specified in Figma.
• [ ] Make sure that "End user experience" sections on each of the pages on Controls > Setup experience page have copy updated as specified in Figma and video (with correct attributes - show controls, loop, etc.) and images as specified in Figma.
• [ ] Make sure that swiftDialog (install software and scripts) is closed after all installs and scripts are finished (success or error) before Apple's local account configuration window pops up
• [ ] Replicate customer-starchik's expected deployment:
  • [ ] Create a bootstrap package that includes fleetd and a script that runs after fleetd is installed.
    • fleetd should be created with fleetctl package --tls_client_cert=path/to/cert and --tls_client_key=path/to/key.
    • The script should lay down a certificate and key (can be fake for QA purposes) at the locations specified in fleetctl package. The script should also set the ORBIT_FLEET_URL fleetd env variable in fleet'd plist (can be fake URL for QA purposes).
• [ ] Add this bootstrap package to Fleet and choose "Install Fleet's agent (fleetd) manually"
• [ ] Enroll a new Mac and check to make sure fleetd is installed and configured
• [ ] Test user experience if fleetd fails to install as part of the bootstrap pkg. Will setup experience get stuck? How does admin get back to a clean state?

Testing notes

Confirmation

1. [ ] Engineer: Added comment to user story confirming successful completion of test plan.
2. [ ] QA: Added comment to user story confirming successful completion of test plan.

[marko-lisica]

DONE: @noahtalerman: https://github.com/fleetdm/fleet/issues/24305#issuecomment-2587680757

---

HeY @noahtalerman, could you bring proposed designs to customer success, so they can check with customer if solution works for them?

[jmwatts]

I accidentally reviewed the test plan on this ticket (didn't have my #g-software filter on) but updated it with a few things. Also was going to mention this ticket is called "Install fleetd with custom certificates during new Mac setup experience" but it seems like what it really accomplishes is the ability to manually install fleetd using a bootstrap package vs having it automatically installed. There are no testing workflows described in this ticket that cover installing fleetd with a custom certificate. Only the ability to choose manual fleetd install vs bootstrap and no automatic fleetd install. Should this ticket be renamed to reflect this?

Will let @PezHub take it from here.

[marko-lisica]

Hey @georgekarrv, heads up, this story is ready to spec.

[noahtalerman]

• [ ] Replicate customer-starchik's expected deployment:
  • [ ] Create a bootstrap package that includes fleetd and a script that runs after fleetd is installed.
    • fleetd should be created with fleetctl package --tls_client_cert=path/to/cert and --tls_client_key=path/to/key.
    • The script should lay down a certificate and key (can be fake for QA purposes) at the locations specified in fleetctl package. The script should also set the ORBIT_FLEET_URL fleetd env variable in fleet'd plist (can be fake URL for QA purposes).
  • [ ] Add this bootstrap package to Fleet and choose "Install Fleet's agent (fleetd) manually"
  • [ ] Enroll a new Mac and check to make sure fleetd is installed and configured

@georgekarrv @PezHub I added the above to the test plan and pulled this one back to user story review. To @jmwatts's point here, I think we want to replicate the customer's expected deployment during QA. We can take a look together during user story review tomorrow.

cc @marko-lisica

[PezHub]

@noahtalerman it would be helpful to get an example bootstrap pkg and script from the customer if possible.

[noahtalerman]

FYI @PezHub @marko-lisica here's more info on starchik's setup: https://docs.google.com/document/d/18d5pnmUZTU4tUvkd65jWQn0_Ypx6MzmZZ0A3Kxm9-l4/edit?tab=t.0

[noahtalerman]

@georgekarrv just a reminder that this user story is ready to spec. Can you please complete the TODOs in the "Engineering" section to get this one ready for estimation?

[noahtalerman]

Hey @georgekarrv just a reminder that this user story is ready to spec and estimation is tomorrow! Can you please complete the TODOs in the "Engineering" and "Test plan" sections?

[noahtalerman]

Hey @georgekarrv just a reminder that this user story is ready to spec and estimation is today!

Can you please work with @PezHub to complete the TODOs in the "Engineering" and the "Test plan" so we can bring this one to estimation?

[georgekarrv]

@marko-lisica will the swift-dialog auto-close still happen if there are any failures?

[marko-lisica]

@marko-lisica will the swift-dialog auto-close still happen if there are any failures?

@georgekarrv Yes, we want to auto-close always, no matter if something failed. It's specified here in Figma.

[nonpunctual]

@marko-lisica @noahtalerman from customer-starchik:

probably want to ensure that the Fleetd configuration profile that gets sent down normally is also excluded when a customer chooses to roll their own package. that way, we can send our own profile with our configs without fear of conflicts

[zayhanlon]

@noahtalerman just for your internal notes, want share that CS believes this issue would help with a massive migration/workflow blocking problem at customer-starchik

[marko-lisica]

@georgekarrv I added missing test plan items and assigned it to you, so you can bring this to an additional estimation for things that we added during expedited drafting (blocking setup experience software and scripts, when fleetd agent is installed manually).

[nonpunctual]

related: https://github.com/fleetdm/fleet/issues/15048

[PezHub]

QA Test Results:

Ran thru the test plan successfully, minus this one particular task:

We did ask the customer to test in their staging env and they reported that their custom bootstrap pkg worked as expected.

Okay tested that scenario successfully.

• confirmed the bootstrap fleetd package is NOT sent by Fleet
• we installed our fleetd package (built via fleetctl) ourselves in our bootstrap logic
• confirmed we get osquery connecting into the server and refresh vitals

Moving to ✅ Ready for Release

[fleet-release]

Custom config blooms, Mac setup enriches Fleet, Enhanced control blooms.

[fleet-release]

Custom config blooms, In the Mac setup's embrace, Fleet now finds new rooms.