Default install script for .exe doesn't work in most cases

[willmayhone88]

Fleet version: 4.52.0, fleetd version 1.27.0

Web browser and operating system: Windows 11 23H2

---

💥  Actual behavior

Uploaded a package (Notion Setup 3.9.3.exe) using the ui. On host device page, selected the software and then selected the install option. Software did not perform the installation, it just copied the exe file to "C:\Program Files\Notion Setup 3.9.3". From there I can install it locally on the device, but the expected behavior would be that it actually performs the installation of the software.

🧑‍💻  Steps to reproduce

1. Add .exe package to fleet
2. Install software via device page

🕯️ More info (optional)

Need to reproduce with other exe files. This behavior does not occur when deploying .msi files. The error may be due to it not properly installing in the user context.

UPDATE: converting this one to feature request, since we need to improve default uninstall script (which is used for cleanup, in case post-install script fails)

🛠️ To fix

Next steps:

1. Update the default install script for .exe to:

# Learn more about .exe install scripts: http://fleetdm.com/learn-more-about/exe-install-scripts

$exeFilePath = "${env:INSTALLER_PATH}"

# Add argument to install silently
# Argument to make install silent depends on installer,
# each installer might use different argument (usually it's "/S" or "/s")
$processOptions = @{
  FilePath = "$exeFilePath"
  ArgumentList = "/S"
  PassThru = $true
  Wait = $true
}

# Start process and track exit code
$process = Start-Process @processOptions
$exitCode = $process.ExitCode

# Prints the exit code
Write-Host "Install exit code: $exitCode"

2. Transform this commentto the article about .exe installation (device-wide, user-wide, and raw executables)
3. In default script: in the comment point to article with redirect (UI links)
4. Test scripts from the comment with following apps: 1Password, Yubi Key Manager, Mozilla Firefox, Notion, FileZilla, Zoom, and Figma
5. Default script redirect: PR

[roperzh]

This is working as designed (not saying we can't improve/change it!)

We didn't find a way to distinguish between exe files that are installers vs exe files that are executable programs.

You can use a custom install script if you want Fleet to treat the exe as an installer, and provide whatever arguments the installer needs as well.

[roperzh]

From @PezHub we used something like this in the past for CrowdStrike

$exeFilePath = "${env:INSTALLER_PATH}"
$installProcess = Start-Process $exeFilePath `
  -ArgumentList "/install /quiet /norestart CID=[license]" `
  -PassThru -Verb RunAs -Wait

[georgekarrv]

Going to chat about how best to tackle this one in Design review tomorrow.

[marko-lisica]

Research for .exe installers

For .exe files there's no unique script/command that will work for all installers as opposed to PKG and MSI. For .msi installers you can run msiexec specify some of parameters that developer documented and it works, similar for .pkg

.exe can be raw execsutable or installer. While working on #14921 we defined default script for raw executables that moves installer to program files only.

What I learned is that in the real word most .exe files are installers. I took 7 "celebrity" apps that are often used in enterprise and tested them. All of them were installers. I tested these apps: 1Password, Yubi Key Manager, Mozilla Firefox, Notion, FileZilla, Zoom, and Figma.

Besides raw and installer .exe files, installers can differ, they can install program device-wide or to the user scope only. There are few ways to learn which type of .exe you're working on. Use google and search "some-installer exe silent install" and find what flag you need to use and usually you can find if it's device-wide or scoped to user.

For device-wide installers, we can use the Start-Process PowerShell cmdlet and it will work.
For user-scoped installers, we need another way, since Fleet runs the install script as SYSTEM, and Start-Process won't be able to install the user-scoped installer.

Based on that there are 2 scripts:

Device-wide installers:

$exeFilePath = "${env:INSTALLER_PATH}"

# Add argument to install silently
# Argument to make install silent depends on installer,
# each installer might use different argument (usually it's "/S" or "/s")
$processOptions = @{
  FilePath = "$exeFilePath"
  ArgumentList = "/S"
  PassThru = $true
  Wait = $true
}

# Start process and track exit code
$process = Start-Process @processOptions
$exitCode = $process.ExitCode

# Prints the exit code
Write-Host "Install exit code: $exitCode"

User-scoped installers:

$exeFilePath = "${env:INSTALLER_PATH}"

# Task properties
# Task will run .exe installer with specified -Argument ("/S" in this case)
# Task will be started by logged user and will run if laptop is running on battery
$action = New-ScheduledTaskAction -Execute "$exeFilePath" -Argument "/S"
$trigger = New-ScheduledTaskTrigger -AtLogOn
$userID = Get-CimInstance -ClassName Win32_ComputerSystem | Select-Object -expand UserName
$principal = New-ScheduledTaskPrincipal -UserId "$userID"
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries

# Create task object with properties defined above
$task = New-ScheduledTask -Action $action -Trigger $trigger -Principal $principal -Settings $settings

Write-Host "Starting ScheduledTask."

# Register and start task
$taskName = "fleetExeInsallation"
Register-ScheduledTask "$taskName" -InputObject $task
Start-ScheduledTask -TaskName "$taskName"

# Check if task is running to unregister
$state = (Get-ScheduledTask -TaskName "$taskName").State

while ($state  -ne "Running") {
  Write-Host "ScheduledTask is '$state'. Waiting to run .exe..."
}

Write-Host "ScheduledTask is '$state'. Waiting for .exe to install..."

# Pause script execution for 2 minutes to give some extra time to run .exe before removing the task
Start-Sleep -Seconds 120

Write-Host "Removing ScheduledTask: $taskName."

# Remove task
Unregister-ScheduledTask -TaskName "$taskName" -Confirm:$false

For instance, the Notion setup is made to run in user scope so we need to use a second script. Same with 1Password (more info here), Zoom and Figma.

I would advise to use MSI whenever possible. I checked and Zoom and 1Password have MSI installers as well, same thing with Slack if you google for it.

Script to install to the user scope is inspired by PSAppDeployToolkit - Execute-ProcessAsUser.

The install script creates a scheduled task that will run by the current (logged-in) user. After it's done task is removed.

NOTE: Each .exe might have a different flag to run silently. (E.g. 1Password: --silent, Notion: /S, Figma: -s)

[marko-lisica]

cc ^^ @noahtalerman @willmayhone88

I think to unblock the customer we can forward them a second script (user-scoped installer). I tested this one with Notion installer and works well.

[marko-lisica]

Hey @georgekarrv, solution is specified in the "To fix" section of issue description. Passing back to engineering.

[mna]

Here are the results of my tests with the new default script (for device-wide .exe install):

• 1Password : reported as installed (installation process succeeded), but is not installed
• Figma: hangs forever, never completes (the Figma Desktop process stays alive on the host, if I kill it, it does report as successfully installed, but is not)
• Filezilla: completes successfully, is actually installed.
• Firefox: completes successfully, is not installed
• Notion: completes successfully, is not installed
• Yubi: completes successfully, is actually installed.
• Zoom: completes successfully, is not installed

And here are the results with the alternative (user-scope) script:

The user-scoped script doesn't actually work for me, it hangs and never completes. Taking a closer look, I think it's because of this loop - if it's not "Running" then it enters a never-ending loop because the state is never refreshed:

# Check if task is running to unregister
$state = (Get-ScheduledTask -TaskName "$taskName").State

while ($state  -ne "Running") {
  Write-Host "ScheduledTask is '$state'. Waiting to run .exe..."
}

I'll try to fix the script.

[mna]

Making slow progress but I think the issue is that when running from fleetd (and unlike when running the script directly in a powershell terminal), the current user differs from the user of the scheduled task (which is created with the logged-in user, while fleetd runs as admin), and that results in Get-ScheduledTask not seeing the task we just registered.

[mna]

Still trying to debug the user-scoped script, it successfully registers the task and gets its state but it never transitions to "started" (to run it immediately):

Installing software...
Failed
Starting ScheduledTask for 'C:\WINDOWS\SystemTemp\3653429915\1PasswordSetup-latest.exe'


TaskPath                                       TaskName                          State     

--------                                       --------                          -----     

\                                              fleetExe1PassInstallationD        Ready     

ScheduledTask registered.
ScheduledTask is 'Ready'
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
ScheduledTask is 'Ready'. Waiting to run .exe...
Timed-out waiting for scheduled task state.

[mna]

I found the issue with running the task - my local Windows user did not have access to C:\WINDOWS\SystemTemp, which is where we store the temporary installer file. Once I visited this directory in the File Explorer, it asked if I wanted to continue and granted me access, and then the task successfully ran (and the software installed in user-scope).

That's a pretty big gotcha to keep in mind for that user-scoped script! @marko-lisica

[mna]

With the fix in place, here are the results with the alternative (user-scope) script:

• 1Password: completes successfully, is actually installed.
• Figma: completes successfully, is actually installed.
• FileZilla: completes successfully, is actually installed.
• Firefox: completes successfully, but is not installed (it did ask for privilege escalation in a window on the host, but I didn't adjust the "quiet" flag in any way so maybe it didn't have the right flag, but still it did not end up installed)
• Notion: completes successfully, is actually installed.
• YubiKey: install fails after timeout, history in the task scheduler shows that it failed to start (not clear what the reason is, but it probably requires to be launched as admin as it works with the device-scoped script).
• Zoom: completes successfully, is actually installed but displayed a UI and asked for privilege escalation, but I didn't adjust the quiet flag).

So all those that did not work with the device-scoped script did succeed and install properly with the user-scoped script.

Two important caveats:

• User must be logged-in for the install to succeed
• User must have access to the temp directory (which it doesn't have by default, at least in my setup)

@marko-lisica FYI, I'll wait for your feedback based on your discussion with Noah on how we move this ticket forward (change the default .exe script or just document the various options and maybe a link to help guide the users, etc.).

[mna]

The updated user-scoped script, with the fixes and faster completion if it takes less than 120 seconds:

$exeFilePath = "${env:INSTALLER_PATH}"

# Task properties
# Task will run .exe installer with specified -Argument ("/S" in this case)
# Task will be started by logged user and will run if laptop is running on battery
$action = New-ScheduledTaskAction -Execute "$exeFilePath" -Argument "/S"
$trigger = New-ScheduledTaskTrigger -AtLogOn
$userName = Get-CimInstance -ClassName Win32_ComputerSystem | Select-Object -expand UserName
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries

# Create task object with properties defined above
$task = New-ScheduledTask -Action $action -Trigger $trigger -Settings $settings

# Register the task
$taskName = "fleetExeInstallation"
Register-ScheduledTask "$taskName" -InputObject $task -User "$userName"

# keep track of start time, to cancel if taking too long to start
$startDate = Get-Date

# Start the task now that it is ready
Start-ScheduledTask -TaskName "$taskName" -TaskPath "\"

# Wait for the task to be running
$state = (Get-ScheduledTask -TaskName "$taskName").State
Write-Host "ScheduledTask is '$state'"

while ($state  -ne "Running") {
  Write-Host "ScheduledTask is '$state'. Waiting to run .exe..."

  $endDate = Get-Date
  $elapsedTime = New-Timespan -Start $startDate -End $endDate
  if ($elapsedTime.TotalSeconds -gt 120) {
    Write-Host "Timed-out waiting for scheduled task state."
    Exit 1
  }

  Start-Sleep -Seconds 1
  $state = (Get-ScheduledTask -TaskName "$taskName").State
}

# Wait for the task to be done
$state = (Get-ScheduledTask -TaskName "$taskName").State
while ($state  -eq "Running") {
  Write-Host "ScheduledTask is '$state'. Waiting for .exe to complete..."

  $endDate = Get-Date
  $elapsedTime = New-Timespan -Start $startDate -End $endDate
  if ($elapsedTime.TotalSeconds -gt 120) {
    Write-Host "Timed-out waiting for scheduled task state."
    Exit 1
  }

  Start-Sleep -Seconds 10
  $state = (Get-ScheduledTask -TaskName "$taskName").State
}

# Remove task
Write-Host "Removing ScheduledTask: $taskName."
Unregister-ScheduledTask -TaskName "$taskName" -Confirm:$false

[mna]

@marko-lisica One additional thing to keep in mind - and that's regardless of what we use for the default .exe install script - is that currently whatever .exe script we use for install, we always use the same removal script (https://github.com/fleetdm/fleet/blob/2923b606da831d1031d5c479e9bb5d30ec268f2f/pkg/file/scripts/remove_exe.ps1). That removal script only makes sense for our simple "copy the .exe to Program Files" default install script.

It might make sense, as a future improvement, to allow customization of the removal script too (which is only ever used if there is a post-install step, so it could be hidden unless a pos-install script is provided). Although one could argue that the post-install script could have a "failure exit" trap that takes care of all the removal, but it makes that post-install more complex (and doesn't prevent our .exe removal script from being executed).

[mna]

Unassigning myself and moving back to Ready, will be converted to a feature request and go through product following the design meeting we had today.

[marko-lisica]

After todays discussion, we decided to convert this to feature request. While working on this bug we learned how most .exe installers work and came up with install scripts that work for both user scope installers and device scope installers. Currently we can't have default uninstall script that work for these installers, because product name is required in order to find UninstallString from registry.

We need to improve this and set new environment variable with product name in order to have use it with uninstall script.

[noahtalerman]

Hey @lukeheath heads up, it looks like this issue got pulled onto confirm and celebrate even though we didn't ship the improvement.

I think that's because we forgot to pull of the 4.55 milestone when we deprioritized the improvement.

I just removed the milestone and removed this from drafting.

[noahtalerman]

While working on this bug we learned how most .exe installers work and came up with install scripts that work for both user scope installers and device scope installers. Currently we can't have default uninstall script that work for these installers, because product name is required in order to find UninstallString from registry.

Hey @marko-lisica will this be addressed as part of the uninstall packages story? #20320

[sharon-fdm]

Hey team! Please add your planning poker estimate with Zenhub @getvictor @lucasmrod @mostlikelee

[sharon-fdm]

We estimated this and think @getvictor can take it as part of the #20320 story since it's the same code area. Taking it for now. @noahtalerman if you think otherwise please advise.

[getvictor]

@marko-lisica For implementation, is there 1 script or 2? Do we default to device-wide installer and expect the customer to find the user-scoped installer in the article?

[marko-lisica]

@marko-lisica For implementation, is there 1 script or 2? Do we default to device-wide installer and expect the customer to find the user-scoped installer in the article?

@getvictor That's right. I would default to device-wide installer and probably we could have "Learn more" link somewhere in the UI (which points to the article). Other solution could be to use script comment to point to the article as described in "To fix" section. cc @RachelElysia

[getvictor]

I found the issue with running the task - my local Windows user did not have access to C:\WINDOWS\SystemTemp, which is where we store the temporary installer file. Once I visited this directory in the File Explorer, it asked if I wanted to continue and granted me access, and then the task successfully ran (and the software installed in user-scope).

That's a pretty big gotcha to keep in mind for that user-scoped script! @marko-lisica

@marko-lisica @noahtalerman Did you have an idea how to solve this issue? I think fleetd needs to store the installer in a directory that all users can access. But could that be a problem if all users have access to the installer? And we probably want to clean it up after install completed.

@mna For user-scoped script, does the user have to online and logged in when Fleet admin clicks the install button? Or is the install simply scheduled for whenever the user logs in?

[mna]

@getvictor

For user-scoped script, does the user have to online and logged in when Fleet admin clicks the install button? Or is the install simply scheduled for whenever the user logs in?

I don't 100% remember, I think it might've been that it had to be online in my tests but I believe there may be some triggers to play with when creating the Task Scheduler so that it either runs immediately (if the user is logged-in) or as soon as they log in.

[getvictor]

@marko-lisica @noahtalerman, I'm still waiting for your response (see above). Also, is this a P2 story that needs to be included in this week's release?

[marko-lisica]

Did you have an idea how to solve this issue? I think fleetd needs to store the installer in a directory that all users can access. But could that be a problem if all users have access to the installer? And we probably want to clean it up after install completed.

@getvictor Sorry I missed the comment. I think if we can temporary store the installer in the location where it's accessible to any user, we should do it. Currently we remove installer right after the installation. I don't see any problem with it. @lukeheath @noahtalerman What do you think, any security concerns?

I don't 100% remember, I think it might've been that it had to be online in my tests but I believe there may be some triggers to play with when creating the Task Scheduler so that it either runs immediately (if the user is logged-in) or as soon as they log in.

I think there's many options for TaskScheduler, it can be run on login, but also there's command to run it manually and remove after (I used that when testing the script).

[noahtalerman]

I think if we can temporary store the installer in the location where it's accessible to any user, we should do it. Currently we remove installer right after the installation. I don't see any problem with it. @lukeheath @noahtalerman What do you think, any security concerns?

Hey @getvictor and @marko-lisica! I think up to @lukeheath (architecture DRI)

Victor, to move quickly, I would get started on your proposed approach (all users can access + cleanup).

Luke, please ping Victor if you don't think this is the right approach. Thanks!

[lukeheath]

I agree with @getvictor's suggested approach.

[valentinpezon-primo]

Hey @lukeheath @mna @getvictor

If it's mature enough, it possible to have the new install script as something I can copy-paste so i can do some test and try it before it's released ?

thanks !

[JoStableford]

Linked to Unthread ticket:

Conversation #2912)

[getvictor]

Hey @lukeheath @mna @getvictor

If it's mature enough, it possible to have the new install script as something I can copy-paste so i can do some test and try it before it's released ?

thanks !

@valentinpezon-primo See the default-machine-scope-install.ps1 and default-user-scope-install.ps1 at https://gist.github.com/getvictor/04eae140a4680944105adce928da12a8