Upload and install tarballs (`.tar.gz` packages) on Linux

[noahtalerman]

Goal

User story
As an admin,
I want to upload .tar.gz and .tgz packages
so that I don't have to host & install these packages using separate tools (ex. Artifactory for hosting & Salt for installing).

Key result

Customer request

Original requests

• #22795

Context

• Product designer: @eugkuo

• @noahtalerman: We can use this .rpm story as a template for wireframes: #20537

Changes

Product

• [ ] UI changes: Figma link
• [ ] CLI (fleetctl) usage changes: No changes
• [ ] YAML changes: Require install/uninstall scripts when specifying tarballs
• [ ] REST API changes: https://github.com/fleetdm/fleet/pull/27202
• [ ] Fleet's agent (fleetd) changes: No changes
• [ ] GitOps mode changes: No changes
• [ ] Activity changes: The current custom packages activity are shown when the package (tar) is added, edited, or deleted
• [ ] Permissions changes: No changes
• [ ] Changes to paid features or tiers: No changes
• [ ] Transparency changes: No changes
• [x] First draft of test plan added
• [ ] Other reference documentation changes: Check for mention of supported package types, in all the reference docs and guides and make sure to add mention of tar.
  • https://fleetdm.com/guides/deploy-software-packages#basic-article
• [ ] Once shipped, requester has been notified
• [ ] Once shipped, dogfooding issue has been filed

Engineering

• [x] Test plan is finalized
• [ ] Feature guide changes: Update "software packages" guides to mention tarballs. May need to revise notes around (un)install scripts.
• [ ] Error messagoe (figma link) links to this URL and need to explain what a valid archive means.

Tarballs don't include package IDs, so we'll need to account for that in the API/DB.

Can build on existing work for EXEs: #27267

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Risk level: Low

Test plan

Main test

1. Navigate to 'Add software > Custom packages'
2. Upload a tar file
3. Ensure that the advanced options open after select is complete and we verify that the attached item is a tarball.
4. Ensure that the Pre-install query,. install script, Post-install script, and Uninstall script areas are empty.
5. Verify that tool tips are added "Install script" and "Uninstall script" to indicate that these are required fields.
6. Ensure that the 'Add software' button is disabled until install script and uninstall script areas are filled out.
7. Add an install script and an uninstall script.
8. Select 'add software'
9. Ensure you are taken to the software detail page.
   1. Ensure Actions > install is disabled. Rolling over it should show a tool tip.
   2. Ensure that 'Hosts' doesn't appear.
10. Select 'back to software'
    1. Ensure tar file shows up in the software list.
    2. Ensure that 'view all hosts' is removed on rollover.

Additional things to test

1. Upload an invalid tar file.
   1. Verify that an error message is displayed.
2. With a sample tar.gz package and install script, verify that the install is successful when deployed to an enrolled host. (Please provide the sample you tested with to the QA engineer)

Testing notes

Confirmation

1. [ ] Engineer: Added comment to user story confirming successful completion of test plan.
2. [ ] QA: Added comment to user story confirming successful completion of test plan.

[iansltx]

So, this is much closer to the .dmg story for macOS than something like rpm/deb, but better/worse effort-wise in a few ways.

Better because

We won't have to wrangle a platform-specific tool like hdiutil; we can pull data out of tar-gz'd files using code we already have.

Worse because

1. From my testing on other tgz'd apps (intentionally not using the word "packages" because they aren't packages...they're just a zip file), these apps won't show up in software inventory because we don't know where to look for them. That means that we won't be able to ship automatically built policy queries for this file format, nor will manual policies be easy to create.
2. .tar.gz files don't have a standard for app metadata, so we won't be able to extract version or app name from the payload. All we can know for sure is that we got something that we can decompress, and maybe that it has some binaries inside.
3. Since these files don't include install instructions (they're a "dumb container" format, to use @nonpunctual's words), we have to decide, or force the admin to specify, where to extract the files to. Since we're talking about Linux, there isn't an obvious "just copy it to /Applications" answer.
4. The above means that we either have to be opinionated on (un)install scripts in a way that's less likely to work than for any other installer package type we support, or force admins to specify, potentially with some placeholders e.g. "$TGZ_FILE_NAME" so they can point tar xf to the right place.

This is in contrast to rpm/deb formats, where the package includes instructions on what to install where, has uninstall instructions built-in, and records that it's been installed in a place where rpm_packages or deb_packages can grab it.

Potential solutions

Bundling into RPMs/DEBs on-upload

To do this, we would prompt for the information required to do build the package as part of the upload process, somewhat similar to the debreate package for .deb files. If we did that we could use the existing software inventory mechanisms to detect installs etc. so policies/auto-install would work.

The big catch here is that we'd need to collect additional fields that we normally just infer from package uploads: package name and version, as well as which directory to extract the tgz to. Potentially in addition to other script lines to run during install/uninstall so we aren't splitting logic between the script inside the .deb/.rpm and our standard (un)install scripts (which is very simple for .deb/.rpm). This would require additional UI specific to tgz uploads, either to directly specify the data or potentially point our own metadata extraction to a file inside the archive to extract data from (not sure if the latter will work). Not a massive implementation lift, but drastically larger design-wise than e.g. rpms were.

The other, much smaller, catch with this is we'd be limiting the tarball to a single family of distros (probably not a problem, and they could just upload the tgz twice with different target formats as we wouldn't be persisting the tgz directly).

Setting up a new source type

To do this, we would add a tgz_packages source that will never match anything in software inventory. Default install script would be "extract the contents of this archive to somedir" and default uninstall script would be "rm -rf somedir". "Is this installed" check could use the file table to check if that directory existed or similar, but I can't think of a reliable way to do an "is this up to date" check.

The catch here is that the scripts would be super naive soif we provided an example script changes would be good that the scripts would get overridden (though placeholders like $TGZ_FILE_NAME may still be useful).

We would also still need to ask the uploader to enter version (or just have a blank version for all tgz uploads) as part of their upload, and would need to ask the uploader to enter a name for the package for deduplication purposes in software_titles. Which...I think these are useful things to let users override for other packages anyway, but if we're uploading tgz files as-is we have to open this can of worms now.

---

Neither of the solutions are impossible, but both require UX changes specific to tgz's for the custom package add (and edit) views; this isn't the near-no-op from a design perspective that RPMs were.

@eugkuo let me know when this is expected to hit design review and I'm happy to hop on then to hash this out further.

[eugkuo]

Added link to the figma file.

Currently, it adds the .tar.gz format to the list of formats in the 'choose a file' area. Will update more as we discuss.

[iansltx]

See this comment for the latest. tl;dr: we'll need to collect more information from the admin somehow, and can automate very little here. We can still support the format, but since it's basically a Linux-flavored zip file batteries aren't included so the admin will need to bring them.

[eugkuo]

@noahtalerman I've added a figma file and test plan to this ticket.

Looking at the YAML it doesn't appear there are changes since we already have it for custom packages.

I've added .tar.gz as an example in the API docs and indicated that install scripts are required for tarballs but I'm not sure what other changes we need on the public API. Also, not sure what changes we need to make in order to display this in the software file list.

We can go over the other todos in design review tomorrow.

[iansltx]

Couple notes:

1. We would populate scripts on-select, not on-upload. Uploads happen after pressing the "Add" button.
2. We should support .tgz and .tar.gz files.
3. For YAML we'll need to require install/uninstall script to be specified.
4. Pre-install query should be blank in Figma.
5. Do we want a warning box above scripts or help text next to the Add button indicating that tarballs require install/uninstall scripts to be set? Or maybe error text on each field? We can limit showing this to tarballs.

[noahtalerman]

We should support .tgz and .tar.gz files.

Agreed. I tweaked the help text. @eugkuo let me know what you think.

For YAML we'll need to require install/uninstall script to be specified.

Hmm, good point. We forgot about this. I don't think it's worth changing this behavior right now just for tarballs. It's really nice to not have to set these for the more common packages (ex. .pkgs)

@eugkuo @iansltx I think let's go with a default script for tarballs that always fails. When the IT admin attempts to install the software, install always fails with an output that says this:

"Failed to install. Please add a custom install and uninstall script."

They'll see this in the Host details > Activity under the failed to install activity and Host details > Software > Show details.

Eugene, on second thought, I think it makes sense to automatically pop open the "Advanced options" for just tarballs. So the IT admin can see that they need to add their own install and uninstall scripts.

[iansltx]

@noahtalerman We can carve out TARs on YAML without affecting the (non-)requirements for other package types. IMO it's better to have a clear error early on of "hey you need to add this" rather than seeming to upload the tarball fine and then erroring out later in the process. That way we're constraining admin behavior in a predictable way, and we can relax that cinstraint later if we find that we can autodetect scripts for some tarballs.

Re: opening advanced options only on tarballs, I interpreted previous notes as already spec'ing that so glad we're on the same page now.

[noahtalerman]

We can carve out TARs on YAML without affecting the (non-)requirements for other package types. IMO it's better to have a clear error early on of "hey you need to add this" rather than seeming to upload the tarball fine and then erroring out later in the process. That way we're constraining admin behavior in a predictable way, and we can relax that cinstraint later if we find that we can autodetect scripts for some tarballs.

@iansltx good call.

FYI @eugkuo sounds like we can just require install and uninstall scripts on tarballs. All other packages can be saved with empty fields.

Related, I think let's stick with our current pattern of red form field text for required fields. For sake of maintaining fewer UI patterns. We can definitely come back to this later.

I'm thinking something like this:

[iansltx]

@noahtalerman Do we want to auto-extract prior to running the install script, or let the install script run tar xf explicitly? The former would allow us to do cleanup actions on the extracted dir, while the latter would avoid potential duplication if the install process is literally "extract the tarball to where you want it to live."

Guessing we want to do the former so we can guarantee that cleanup happens, but we need to make (and document) a decision one way or the other.

[iansltx]

Of note on the above, if a customer wants "just extract this to X location" then they can have a mv or cp -r as the install script, so if we auto-extract or don't we're not blocking any significant workflows from what I can tell.

[iansltx]

A couple other things we autodetect but won't be able to with the same precision:

1. Software title name. We can use filename without extension but that's not going to match any versions. But we can at least stay out of the way of inventoried software by having a special tar_packages source that doesn't match anything from osquery, nor will it conflict with our bundle ID based unique key since Linux doesn't have bundle IDs.
2. Version. We won't extract version, but that's fine since we already have handling for that now.

One nice thing is that we can validate that the upload file is actually a .tar.gz; as mentioned in the customer call .deb files are just tarballs (or sometimes .tar.zst files) so we can hook into that code.

[iansltx]

As far as future state of matching packages to scripts goes, that feels like something we would solve for common packages with Fleet maintained Apps for Linux, rather than populating scripts on custom package upload, since by taking that approach we can e.g. automatically bundle apps into deb/rpms, which would give us inventory for free. That also sidesteps the question of how we'd detect which script to use, since on initial file selection we only have filename and maybe file size to go off of, both of which are rather brittle.

[iansltx]

Actually, for parsing out title and version, we could check for a regex of [title]-[x.y.z[.a]].[ext] in the filename and split out title and version from that if there's a match. This won't match the customer sample packages as-is, but they could be renamed to match if getting title/version was important enough to the admin.

[eugkuo]

1. We would populate scripts on-select, not on-upload. Uploads happen after pressing the "Add" button.

Updated this in the test plan.

2. We should support .tgz and .tar.gz files.

I see that @noahtalerman added this to the figma. I added this to the API doc changes.

3. For YAML we'll need to require install/uninstall script to be specified.

Hm. Not exactly sure how / what to add here. Will bring it up during design review.

4. Pre-install query should be blank in Figma.

I've blanked this out. Is this a required field as well?

5. Do we want a warning box above scripts or help text next to the Add button indicating that tarballs require install/uninstall scripts to be set? Or maybe error text on each field? We can limit showing this to tarballs.

I'm proposing we add 'required' text to the headers for these two fields and that we grey out the ability to 'add' until these are populated.

@iansltx

[eugkuo]

Related, I think let's stick with our current pattern of red form field text for required fields. For sake of maintaining fewer UI patterns. We can definitely come back to this later.

I'm thinking something like this

@noahtalerman Isn't this the error state after someone has tried to move forward without entering something? Do we have a state that tells people in advance that something is required? Or is it the same.

I've added this for now but will bring it up during design review.

[iansltx]

4. Pre-install query should be blank in Figma.

I've blanked this out. Is this a required field as well?

It is not. Just making sure Figma designs don't show it because tarballs as a rule won't show up in inventory.

[eugkuo]

I've added

• #27330 (draft redirect PR)

to the additional docs section to track this for now.

[jmwatts]

Possible installation test: Pycharm tar xzf pycharm-professional-2024.3.5-aarch64.tar.gz -C /opt/ && cd /opt/pycharm-2024.3.5/bin && ./pycharm.sh (This will launch pycharm after installing so it wouldn't be a "silent" install, but maybe good for testing purposes)

[iansltx]

@jmwatts We'll actually do the tar xzf part in orbit and then nevigate to the extracted dir to run the script, so I think the script would just be

./bin/pycharm.sh

but TBD

[eugkuo]

@noahtalerman @iansltx @mostlikelee

There was some chatter in the figma about 'auto-install' and disabling it with a tooltip to indicate why. I asked @RachelElysia what we use for .exe and I tweaked it to read '.tar.gz' archives. Please take a look and let us know if it's accurate. Figma link, copy below

Fleet can’t create a policy to detect existing installations for .tar.gz archives. To automatically install .tar.gz archives, add a custom policy and enable the install software automation on the Policies page.

[mostlikelee]

@eugkuo looks good to me! Good to be explicit that tar.gz has the same limitations as exe installers.

[eugkuo]

Thanks @mostlikelee ! I've resolved the comment in figma.

[iansltx]

@jmwatts (plus @lucasmrod @xpkoala because y'all have likely tested this workflow before explicitly) The fleetd code on this, to avoid significant duplication of .tar.gz extraction code, touches the same code TUF updates use to extract tarballs. We have test coverage for this code, but due to time constraints over the past week and a half I haven't run through a manual workflow on updates from the new version to a newer one involving tarball downloads, and we should do this to make sure I didn't break something.

I mention "from the new version" since the revised extraction code (basically just extracting part of the function out) would only be in the path for TUF updates once deployed, so we'd want to bump Orbit up to main, then bump up from there to a newer version with some trivial change to make sure the update process continues to work (assuming Orbit updates traverse the .tar.gz code).

[lucasmrod]

@jmwatts (plus @lucasmrod @xpkoala because y'all have likely tested this workflow before explicitly) The fleetd code on this, to avoid significant duplication of .tar.gz extraction code, touches the same code TUF updates use to extract tarballs. We have test coverage for this code, but due to time constraints over the past week and a half I haven't run through a manual workflow on updates from the new version to a newer one involving tarball downloads, and we should do this to make sure I didn't break something.

I mention "from the new version" since the revised extraction code (basically just extracting part of the function out) would only be in the path for TUF updates once deployed, so we'd want to bump Orbit up to main, then bump up from there to a newer version with some trivial change to make sure the update process continues to work (assuming Orbit updates traverse the .tar.gz code).

Thanks for pointing this out!

Yes, it's a must to test auto-update from "N+1" to "N+2" (on the three OSs).

The components that are currently tar.gz are:

• Fleet Desktop, osquery, swiftDialog, and Nudge for macOS.
• Fleet Desktop for Linux.

(orbit itself is just an executable, so no tar.gz)

That said:

• swiftDialog does not come built into the package so it gets downloaded on macOS automatically now (starting in 1.42.0), so nothing auto-update-related to test there just checking that they eventually download and extract on the device is enough.
• Nudge will be tested in this other fix: https://github.com/fleetdm/fleet/issues/28727.

So what needs to be tested auto-update-related for this issue is: an auto-update of "Fleet Desktop" and "osquery" on macOS and "Fleet Desktop" on Linux.

[jmwatts]

@iansltx I'm getting this when attempting to install a valid tar.gz:

2025-05-06T12:25:31.383471-05:00 ubuntu-linux-2404 orbit[175304]: 2025-05-06T12:25:31-05:00 ERR failed extract .tar.gz archive error="extract \"/tmp/2091266637/go1.24.2.linux-arm64.tar.gz\": mkdir \"go/\": Path /tmp/2091266637/extracted already exists with mode 20000000700 instead of the expected 20000000755" installerID=92468f29-9e31-4d17-ab2b-1ee55d2ffdb0 runner=installer

[lucasmrod]

Path /tmp/2091266637/extracted already exists with mode 20000000700 instead of the expected 20000000755"

This is a security check added to TUF downloads. Seems related to https://github.com/fleetdm/fleet/issues/25775. @jmwatts What's the output of your umask?

[iansltx]

@lucasmrod Yeah, I backed out that check at the beginning of the call (for installs only; kept it for TUF) but didn't account for needing to create additional directories as part of extraction. Either I'm trying to solve this the wrong way or we may need to split out the extraction paths entirely so I'm not adding too many paths into critical code.

[lucasmrod]

@lucasmrod Yeah, I backed out that check at the beginning of the call (for installs only; kept it for TUF) but didn't account for needing to create additional directories as part of extraction. Either I'm trying to solve this the wrong way or we may need to split out the extraction paths entirely so I'm not adding too many paths into critical code.

To decrease risk (of breaking auto-updates) I'm ok if we duplicate code here. (Copy/paste from TUF + removing that thing.)

[iansltx]

Most recent PR picks code duplication (plus some additional checks) over doing anything with code that touches updates. Confirmed that with the latest PR any changes touching update.go are fully backed out, so once that's merged test surface no longer includes N+1 and N+2 TUF updates, and risk is significantly lower.

[iansltx]

TODO for me: instuctions on creating an invalid tarball that still gets detected as a tarball