tern icon indicating copy to clipboard operation
tern copied to clipboard

Published 20 hours ago • verified publisher icon tern-tools

Packages installed by scripts are not reported

Open dotcarls opened this issue 4 years ago • 5 comments

First off, I want to say this project is awesome -- I spent the last day playing around with it and am very excited to (hopefully!) contribute.

Describe the bug Some projects utilize a shell script to perform package installation / configuration rather than inline statements in the Dockerfile. Similarly, some projects provide an 'easy install' method to install / configure the project's package / dependencies. Packages installed in this manner are not reported by tern.

While testing this, I noticed what appears to be an edge case related to #743 / #756 -- install commands that are not 'directly' executed in a RUN directive are not identified.

To Reproduce Below are three examples where the package hello is installed on top of the debian:buster-20200803-slim image.

Case 1: A local script hello_script_file:

FROM debian:buster-20200803-slim
COPY ./hello_install.sh /tmp/hello_install.sh
RUN /tmp/hello_install.sh

hello_install.sh:

#!/usr/bin/env sh
apt-get update && apt-get -y install hello

tern output for layer:

------------------------------------------------

	Layer 3:
		info: Instruction Line: RUN /tmp/hello_install.sh # buildkit
		warning: 
Unrecognized Commands:/tmp/hello_install.sh # buildkit

	File licenses found in Layer:  None
	Packages found in Layer:  None
	Licenses found in Layer:  None
------------------------------------------------

Case 2: A script fetched with curl hello_script_curl:

FROM debian:buster-20200803-slim
RUN apt-get update && apt-get -y install curl
RUN curl -o- https://gist.githubusercontent.com/dotcarls/42bcce3e83e7aa00f35feaf945d16f4d/raw/a9e5243a3a6666b6e72c079293011023e897b485/hello_install | sh

hello_install gist:

#!/usr/bin/env sh
apt-get update && apt-get -y install hello

tern output for layer:

------------------------------------------------

	Layer 3:
		info: Instruction Line: RUN curl -o- https://gist.githubusercontent.com/dotcarls/42bcce3e83e7aa00f35feaf945d16f4d/raw/a9e5243a3a6666b6e72c079293011023e897b485/hello_install | sh # buildkit
		warning: 
Unrecognized Commands:curl -o- https://gist.githubusercontent.com/dotcarls/42bcce3e83e7aa00f35feaf945d16f4d/raw/a9e5243a3a6666b6e72c079293011023e897b485/hello_install | sh # buildkit

	File licenses found in Layer:  None
	Packages found in Layer:  None
	Licenses found in Layer:  None
------------------------------------------------

Case 3: An echo'd script: I'm not familiar enough with the code to say this is really a bug, but I it seems to me like #743 / #756 is trying to parse RUN directives for package install commands. Please forgive me if I've misunderstood!

hello_script_echo:

FROM debian:buster-20200803-slim
RUN echo "#!/usr/bin/env sh\n apt-get update && apt-get -y install hello" > /tmp/hello_install.sh && \
    chmod +x /tmp/hello_install.sh && \
    /tmp/hello_install.sh

tern output for layer:

------------------------------------------------

	Layer 2:
		info: Instruction Line: RUN echo "#!/usr/bin/env sh\n apt-get update && apt-get -y install hello" > /tmp/hello_install.sh &&     chmod +x /tmp/hello_install.sh &&     /tmp/hello_install.sh # buildkit
		warning: 
Unrecognized Commands:echo #!/usr/bin/env sh\n apt-get update && apt-get -y install hello > /tmp/hello_install.sh
chmod +x /tmp/hello_install.sh
/tmp/hello_install.sh # buildkit

	File licenses found in Layer:  None
	Packages found in Layer:  None
	Licenses found in Layer:  None
------------------------------------------------

Expected behavior All installed packages that are managed by the package managers in tern/command_lib/snippets.yml / tern/command_lib/base.yml should be reported by tern as per Guiding Principle #1. Ideally, the above cases would produce reports similar to the following:

hello_inline:

FROM debian:buster-20200803-slim
RUN apt-get update && apt-get -y install hello

tern output for layer:

------------------------------------------------

	Layer 2:
		info: Instruction Line: RUN apt-get update && apt-get -y install hello # buildkit
		warning: 
Ignored Commands:apt-get update

		info: Layer created by commands: RUN /bin/sh -c apt-get update && apt-get -y install hello # buildkit
		info: Retrieved by invoking listing in command_lib/base.yml
versions:
	in container:
	pkgs=`dpkg --get-selections | cut -f1 -d':' | awk '{print $1}'`
	for p in $pkgs; do dpkg -l $p | awk 'NR>5 {print $3}'; done


names:
	in container:
	dpkg --get-selections | cut -f1 -d':' | awk '{print $1}'

copyrights:
	in container:
	pkgs=`dpkg --get-selections | cut -f1 -d':' | awk '{print $1}'`
	for p in $pkgs; do /bin/cat /usr/share/doc/$p/copyright; echo LICF; done



	Invoking commands from command_lib/base.yml:
		warning: No listing method for 'srcs'. Additional analysis may be required.
No listing method for 'licenses'. Additional analysis may be required.

	File licenses found in Layer:  None
	Packages found in Layer:  hello-2.10-2
	Licenses found in Layer:  None
------------------------------------------------

Environment you are running Tern on Enter all that apply

  • Output of 'tern --version'
root@b01c5367a39b:/src# tern --version
Tern at commit f752cde11f8f5243d0965b6e53dc906b58773dcf
   python version = 3.7.3 (default, Jul 25 2020, 13:03:44)

  • Operating System (Linux Distro and version or Mac or Windows) Host is macOS Catalina 10.15.5.

  • Container OS Container is debian:buster and is based on the Dockerfile in this repo. I made the following change:

ADD . /src
WORKDIR /src

RUN pip3 install wheel && \
    pip3 install -e.[dev]
  • Python version (3.6 or higher)
root@b01c5367a39b:/src# python3 --version
Python 3.7.3

dotcarls avatar Aug 08 '20 07:08 dotcarls

Regarding a solution, I believe that relying on heuristics to detect package installation is insufficient for all use-cases. It'd be great to have a flag like --full-audit that would force tern to check for packages after each and every RUN layer rather than only those where tern can parse a package installation command.

dotcarls avatar Aug 08 '20 07:08 dotcarls

Hi @dotcarls, thanks for opening this issue and for your interest in Tern! We are always welcoming new contributors and would love to see you get involved :)

As you have correctly stated, Tern does not currently detect packages installed by scripts. The reason for this right now is mainly the fact that there are infinite ways that people install packages in containers. You have given examples for 3 ways above but the list goes on and parsing for all the ways is not feasible. We have tried to focus on the most common/popular way that packages are installed, i.e. by parsing the RUN commands for common install directives and package manager utilities. This behavior in Tern is also in line with the best practices that we advocate for surrounding container compliance -- using a package manager to install packages is the best way to produce secure, recent and reproducible container images.

That being said, you bring up a valid point that people will find all sorts of interesting ways to install packages and as the project grows, it makes sense to consider these. While I don't think it's worthwhile for Tern to parse RUN commands for install scripts (there's too many variables when it comes to scripts), I like your idea about some sort of feature that could check for packages after each and every RUN layer regardless of install directives found. A couple issues that would need to be addressed prior are 1) Performance - There are currently 9 (soon to be 10) different supported package managers in tern/command_lib/base.yml and running all of these for each RUN layer would definitely be a performance hit, especially for larger images. 2) How to deal with the lack of availability for each package manager by layer. For example, if we mount a layer where a script installs a set of packages using apt, we as humans know by looking at the script that the apt package manager is available in the layer. But if we try to run the other 8-9 package managers to collect package information in that layer, we are going to have to deal with the failures of running all the commands that try to collect the names, versions, licenses, proj_urls and copyrights for each of the 8-9 other package manager utilities not available in the layer. This is another reason why Tern tries to parse the RUN commands looking for a package manager -- so that it only runs commands from base.yml where it knows the utility will be available in the layer.

Thoughts?

FYI -->

Case 3: An echo'd script: I'm not familiar enough with the code to say this is really a bug, but I it seems to me like #743 / #756 is trying to parse RUN directives for package install commands. Please forgive me if I've misunderstood!

You are correct about what #743 / #756 is trying to do here. Although, I don't know if I would qualify this as a "bug" since we don't actually try to parse RUN commands for echo/install script behavior at the moment.

rnjudge avatar Aug 11 '20 01:08 rnjudge

Thanks for the detailed response @rnjudge ! Totally agree with regards to the points about best practices / the infeasibility of writing parsers to cover all installation possibilities.

  1. Performance - There are currently 9 (soon to be 10) different supported package managers in tern/command_lib/base.yml and running all of these for each RUN layer would definitely be a performance hit, especially for larger images.

My expertise is mostly around dpkg (and related apt, apt-get, aptitude, etc.) and npm. My first thought is that tern wouldn't need to execute each package manager on each layer as the list should be filtered to only those where the required binary exists. This would also apply to your 2nd point regarding handling errors -- if which dpkg (or an equivalent check like stating the well known binary path) doesn't yield a usable binary, simply don't run that analysis. This check would have to be repeated for each layer but I think it would be pretty inexpensive.

I'm not sure if this is applicable for other package managers, but another optimization for dpkg based package installations would to have tern check for modifications to the dpgk status / state files. There's a really cool project called dive that presents you with a nice TUI file tree for each layer in an image. This is what a report looks like for the hello_curl example I provided:

Screen Shot 2020-08-11 at 4 48 55 PM

This shows what I believe would be a fairly reliable heuristic for detecting package changes via dpkg -- modifications to files under /var/lib/dpkg indicate a change in package state. I'm not sure what the equivalent would be (or if there is one) for the others like [t]dnf, yum, etc., but I'd be happy to do some investigation.

Implementation wise, my understanding is that tern already collects a list of all files for a layer, but I'm not sure what would be required to add metadata to these lists to track things like modifications. I do think deepening the analysis of the contents of each layer would be very powerful for tern. Consider a somewhat contrived example like:

FROM my-npm-build-image:latest 
COPY ./package*.json /src
RUN npm install --production

FROM my-minimal-node-runtime-image:latest
COPY --from=my-npm-build-image:latest /src/node_modules /app/node_modules

In this case, I don't think tern would have any knowledge of my-npm-build-image:latest. I think the only way it could then report npm packages would be to look into the layer contents to try to extract package metadata from the node_modules directory. I will certainly concede that maintaining this level of introspection across all the supported package managers might be unrealistic (and probably out of scope for this issue report), but I'd love to hear your thoughts.

dotcarls avatar Aug 12 '20 00:08 dotcarls

My expertise is mostly around dpkg (and related apt, apt-get, aptitude, etc.) and npm. My first thought is that tern wouldn't need to execute each package manager on each layer as the list should be filtered to only those where the required binary exists. This would also apply to your 2nd point regarding handling errors -- if which dpkg (or an equivalent check like stating the well known binary path) doesn't yield a usable binary, simply don't run that analysis. This check would have to be repeated for each layer but I think it would be pretty inexpensive.

@dotcarls Agree that this would be a better way to do it without taking huge performance hit. We actually already do this for the base layer using the get_base_bin() function located in tern/analyze/common.py. While this function only checks for the existence of a single package manager binary, we could do something similar to check for multiple binaries in subsequent layers.

I'm not sure if this is applicable for other package managers, but another optimization for dpkg based package installations would to have tern check for modifications to the dpgk status / state files. There's a really cool project called dive that presents you with a nice TUI file tree for each layer in an image.

I will take a look at this project, thanks for pointing me to it!

This shows what I believe would be a fairly reliable heuristic for detecting package changes via dpkg -- modifications to files under /var/lib/dpkg indicate a change in package state. I'm not sure what the equivalent would be (or if there is one) for the others like [t]dnf, yum, etc., but I'd be happy to do some investigation.

More investigation on whether other package managers support this would be great! If it's only dpkg that supports this type of detection, I would be slightly more hesitant to implement such big functionality around it as I would prefer to find a detection method that works more universally with the other package managers.

Implementation wise, my understanding is that tern already collects a list of all files for a layer, but I'm not sure what would be required to add metadata to these lists to track things like modifications.

Tern does collect a list of all files for a layer. Adding metadata to these file objects to track things like modifications would likely happen in the file_data class.

I do think deepening the analysis of the contents of each layer would be very powerful for tern. Consider a somewhat contrived example like:

I labeled your two Dockerfile examples so they are easier to reference in my response below :)

Dockerfile A:

FROM my-npm-build-image:latest 
COPY ./package*.json /src
RUN npm install --production

Dockerfile B:

FROM my-minimal-node-runtime-image:latest
COPY --from=my-npm-build-image:latest /src/node_modules /app/node_modules

In this case, I don't think tern would have any knowledge of my-npm-build-image:latest. I think the only way it could then report npm packages would be to look into the layer contents to try to extract package metadata from the node_modules directory. I will certainly concede that maintaining this level of introspection across all the supported package managers might be unrealistic (and probably out of scope for this issue report), but I'd love to hear your thoughts.

When Tern runs, it will look for a package manager binary in the first image layer by checking for the existence of the various package manager paths in base.yml (see get_base_bin in tern/analyze/common.py). So for Dockerfile A and B, as long as my-npm-build-image:latest or my-minimal-node-runtime-image:latest had one of the supported package manager paths existing in the first layer filesystem, Tern would find package information in that base layer by running the commands in base.yml under the found package manager binary. This functionality seems similar to what you are proposing above -- checking for a package manger utility in the layer and then running the necessary commands accordingly. If no package manager binary existed, then yes, Tern would not find anything.

For Dockerfile B, you are also right that Tern would not find any packages in the second layer since they are not being installed by a RUN command. In Dockerfile A, however, Tern would recognize the npm install snippet in the RUN command and run the appropriate package collection utilities for that layer.

rnjudge avatar Aug 13 '20 04:08 rnjudge

More investigation on whether other package managers support this would be great! If it's only dpkg that supports this type of detection, I would be slightly more hesitant to implement such big functionality around it as I would prefer to find a detection method that works more universally with the other package managers.

I looked into this and it seems that there are equivalents for all OS variants in base.yml. It boils down to checking the local package database for rpm, dpkg, apk, and pacman. I wrote a script to verify this, it does the following:

  1. From a base image for the OS, stat files in the package database directory
  2. Install nano
  3. stats the package database directory files again
  4. diff the before / after stat output
                                                                       Diff for fedora
                                                                       Before <-> After
2020-07-09 06:48:30.000000000 +0000 790528 /var/lib/rpm/Basenames              |  2020-08-16 22:10:17.060160000 +0000 790528 /var/lib/rpm/Basenames
2020-07-09 06:48:30.000000000 +0000 8192 /var/lib/rpm/Conflictname             |  2020-08-16 22:10:17.064160000 +0000 8192 /var/lib/rpm/Conflictname
2020-07-09 06:48:30.000000000 +0000 204800 /var/lib/rpm/Dirnames               |  2020-08-16 22:10:17.066160000 +0000 204800 /var/lib/rpm/Dirnames
2020-07-09 06:48:36.000000000 +0000 12288 /var/lib/rpm/Group                   |  2020-08-16 22:10:17.062160000 +0000 12288 /var/lib/rpm/Group
2020-07-09 06:48:36.000000000 +0000 12288 /var/lib/rpm/Installtid              |  2020-08-16 22:10:17.066160000 +0000 12288 /var/lib/rpm/Installtid
2020-07-09 06:48:36.000000000 +0000 16384 /var/lib/rpm/Name                    |  2020-08-16 22:10:17.057160000 +0000 16384 /var/lib/rpm/Name
2020-07-09 06:48:36.000000000 +0000 4751360 /var/lib/rpm/Packages              |  2020-08-16 22:10:17.056160000 +0000 4775936 /var/lib/rpm/Packages
2020-07-09 06:48:36.000000000 +0000 81920 /var/lib/rpm/Providename             |  2020-08-16 22:10:17.064160000 +0000 81920 /var/lib/rpm/Providename
2020-07-09 06:48:30.000000000 +0000 57344 /var/lib/rpm/Requirename             |  2020-08-16 22:10:17.063160000 +0000 57344 /var/lib/rpm/Requirename
2020-07-09 06:48:36.000000000 +0000 20480 /var/lib/rpm/Sha1header              |  2020-08-16 22:10:17.067160000 +0000 20480 /var/lib/rpm/Sha1header
2020-07-09 06:48:30.000000000 +0000 20480 /var/lib/rpm/Sigmd5                  |  2020-08-16 22:10:17.067160000 +0000 20480 /var/lib/rpm/Sigmd5
2020-07-09 06:48:36.000000000 +0000 311296 /var/lib/rpm/__db.001               |  2020-08-16 22:10:18.487160000 +0000 311296 /var/lib/rpm/__db.001
2020-07-09 06:48:36.000000000 +0000 90112 /var/lib/rpm/__db.002                |  2020-08-16 22:10:18.486160000 +0000 90112 /var/lib/rpm/__db.002
2020-07-09 06:48:36.000000000 +0000 263800 /var/lib/rpm/__db.003               |  2020-08-16 22:10:18.488160000 +0000 1318912 /var/lib/rpm/__db.003

                                                                       Diff for photon
                                                                       Before <-> After
2020-08-13 00:54:04.000000000 +0000 53248 /var/lib/rpm/Basenames               |  2020-08-16 21:09:14.361722000 +0000 57344 /var/lib/rpm/Basenames
2020-08-13 00:54:04.000000000 +0000 20480 /var/lib/rpm/Dirnames                |  2020-08-16 21:09:14.363722000 +0000 20480 /var/lib/rpm/Dirnames
2020-08-13 00:54:10.000000000 +0000 8192 /var/lib/rpm/Group                    |  2020-08-16 21:09:14.361722000 +0000 8192 /var/lib/rpm/Group
2020-08-13 00:54:10.000000000 +0000 8192 /var/lib/rpm/Installtid               |  2020-08-16 21:09:14.363722000 +0000 8192 /var/lib/rpm/Installtid
2020-08-13 00:54:10.000000000 +0000 8192 /var/lib/rpm/Name                     |  2020-08-16 21:09:14.359722000 +0000 8192 /var/lib/rpm/Name
2020-08-13 00:54:10.000000000 +0000 425984 /var/lib/rpm/Packages               |  2020-08-16 21:09:14.358722000 +0000 450560 /var/lib/rpm/Packages
2020-08-13 00:54:10.000000000 +0000 32768 /var/lib/rpm/Providename             |  2020-08-16 21:09:14.362722000 +0000 32768 /var/lib/rpm/Providename
2020-08-13 00:54:04.000000000 +0000 20480 /var/lib/rpm/Requirename             |  2020-08-16 21:09:14.362722000 +0000 20480 /var/lib/rpm/Requirename
2020-08-13 00:54:10.000000000 +0000 8192 /var/lib/rpm/Sha1header               |  2020-08-16 21:09:14.364722000 +0000 8192 /var/lib/rpm/Sha1header
2020-08-13 00:54:04.000000000 +0000 8192 /var/lib/rpm/Sigmd5                   |  2020-08-16 21:09:14.363722000 +0000 8192 /var/lib/rpm/Sigmd5
                                                                               >  2020-08-16 21:09:14.320722000 +0000 376832 /var/lib/rpm/__db.001
                                                                               >  2020-08-16 21:09:14.338722000 +0000 98304 /var/lib/rpm/__db.002
                                                                               >  2020-08-16 21:09:14.358722000 +0000 684352 /var/lib/rpm/__db.003


                                                                       Diff for debian
                                                                       Before <-> After
2020-08-03 07:00:00.000000000 +0000 4096 /var/lib/dpkg/alternatives            |  2020-08-16 22:10:26.216160000 +0000 4096 /var/lib/dpkg/alternatives
2020-08-03 07:00:00.000000000 +0000 20480 /var/lib/dpkg/info                   |  2020-08-16 22:10:26.188160000 +0000 4096 /var/lib/dpkg/info
2020-08-03 07:00:00.000000000 +0000 0 /var/lib/dpkg/lock                       |  2020-08-16 22:10:26.202160000 +0000 0 /var/lib/dpkg/lock
2020-08-03 07:00:00.000000000 +0000 76300 /var/lib/dpkg/status                 |  2020-08-16 22:10:26.219160000 +0000 77532 /var/lib/dpkg/status
2020-08-03 07:00:00.000000000 +0000 76304 /var/lib/dpkg/status-old             |  2020-08-16 22:10:26.193160000 +0000 77510 /var/lib/dpkg/status-old
2020-08-03 07:00:00.000000000 +0000 4096 /var/lib/dpkg/updates                 |  2020-08-16 22:10:26.221160000 +0000 4096 /var/lib/dpkg/updates


                                                                       Diff for alpine
                                                                       Before <-> After
2020-05-29 14:20:33.000000000 11895 /lib/apk/db/installed                      |  2020-08-16 22:10:28.000000000 16224 /lib/apk/db/installed
2020-05-29 14:20:33.000000000 10752 /lib/apk/db/scripts.tar                    |  2020-08-16 22:10:28.000000000 10752 /lib/apk/db/scripts.tar
2020-05-29 14:20:33.000000000 76 /lib/apk/db/triggers                          |  2020-08-16 22:10:28.000000000 76 /lib/apk/db/triggers

                                                                       Diff for centos
                                                                       Before <-> After
2020-08-09 21:39:32.000000000 +0000 950272 /var/lib/rpm/Basenames              |  2020-08-16 22:10:47.848531000 +0000 950272 /var/lib/rpm/Basenames
2020-08-09 21:39:31.000000000 +0000 8192 /var/lib/rpm/Conflictname             |  2020-08-16 22:10:47.851531000 +0000 8192 /var/lib/rpm/Conflictname
2020-08-09 21:39:32.000000000 +0000 286720 /var/lib/rpm/Dirnames               |  2020-08-16 22:10:47.853531000 +0000 286720 /var/lib/rpm/Dirnames
2020-08-09 21:39:32.000000000 +0000 8192 /var/lib/rpm/Group                    |  2020-08-16 22:10:47.850531000 +0000 8192 /var/lib/rpm/Group
2020-08-09 21:39:32.000000000 +0000 12288 /var/lib/rpm/Installtid              |  2020-08-16 22:10:47.853531000 +0000 12288 /var/lib/rpm/Installtid
2020-08-09 21:39:32.000000000 +0000 20480 /var/lib/rpm/Name                    |  2020-08-16 22:10:47.844531000 +0000 20480 /var/lib/rpm/Name
2020-08-09 21:39:32.000000000 +0000 18944000 /var/lib/rpm/Packages             |  2020-08-16 22:10:47.844531000 +0000 18944000 /var/lib/rpm/Packages
2020-08-09 21:39:32.000000000 +0000 1798144 /var/lib/rpm/Providename           |  2020-08-16 22:10:47.851531000 +0000 1798144 /var/lib/rpm/Providename
2020-08-09 21:39:32.000000000 +0000 86016 /var/lib/rpm/Requirename             |  2020-08-16 22:10:47.851531000 +0000 86016 /var/lib/rpm/Requirename
2020-08-09 21:39:32.000000000 +0000 24576 /var/lib/rpm/Sha1header              |  2020-08-16 22:10:47.854531000 +0000 24576 /var/lib/rpm/Sha1header
2020-08-09 21:39:32.000000000 +0000 16384 /var/lib/rpm/Sigmd5                  |  2020-08-16 22:10:47.854531000 +0000 16384 /var/lib/rpm/Sigmd5
2020-08-09 21:39:32.000000000 +0000 352256 /var/lib/rpm/__db.001               |  2020-08-16 22:10:48.313531000 +0000 352256 /var/lib/rpm/__db.001
2020-08-09 21:39:32.000000000 +0000 98304 /var/lib/rpm/__db.002                |  2020-08-16 22:10:48.290531000 +0000 98304 /var/lib/rpm/__db.002
2020-08-09 21:39:32.000000000 +0000 1318912 /var/lib/rpm/__db.003              |  2020-08-16 22:10:48.294531000 +0000 1318912 /var/lib/rpm/__db.003

                                                                       Diff for opensuse
                                                                       Before <-> After
2020-08-16 07:54:21.000000000 +0000 143360 /var/lib/rpm/Basenames              |  2020-08-16 22:59:58.774335000 +0000 143360 /var/lib/rpm/Basenames
2020-08-16 07:54:21.000000000 +0000 57344 /var/lib/rpm/Dirnames                |  2020-08-16 22:59:58.775335000 +0000 57344 /var/lib/rpm/Dirnames
2020-08-16 07:54:21.000000000 +0000 8192 /var/lib/rpm/Group                    |  2020-08-16 22:59:58.774335000 +0000 8192 /var/lib/rpm/Group
2020-08-16 07:54:21.000000000 +0000 8192 /var/lib/rpm/Installtid               |  2020-08-16 22:59:58.775335000 +0000 8192 /var/lib/rpm/Installtid
2020-08-16 07:54:21.000000000 +0000 8192 /var/lib/rpm/Name                     |  2020-08-16 22:59:58.772335000 +0000 8192 /var/lib/rpm/Name
2020-08-16 07:54:21.000000000 +0000 4042752 /var/lib/rpm/Packages              |  2020-08-16 22:59:58.771335000 +0000 4083712 /var/lib/rpm/Packages
2020-08-16 07:54:21.000000000 +0000 1757184 /var/lib/rpm/Providename           |  2020-08-16 22:59:58.775335000 +0000 1757184 /var/lib/rpm/Providename
2020-08-16 07:54:21.000000000 +0000 57344 /var/lib/rpm/Requirename             |  2020-08-16 22:59:58.774335000 +0000 57344 /var/lib/rpm/Requirename
2020-08-16 07:54:21.000000000 +0000 16384 /var/lib/rpm/Sha1header              |  2020-08-16 22:59:58.775335000 +0000 16384 /var/lib/rpm/Sha1header
2020-08-16 07:54:21.000000000 +0000 16384 /var/lib/rpm/Sigmd5                  |  2020-08-16 22:59:58.775335000 +0000 16384 /var/lib/rpm/Sigmd5

                                                                       Diff for archlinux
                                                                       Before <-> After
                                                                               >  2020-08-16 22:58:41.182530000 +0000 4096 /var/lib/pacman/local/nano-5.0-1

So it seems like there should be a pretty reliable and consistent way to detect package changes based on changes to the filesystem. Now the ~hard~ fun part -- making tern aware of this information so it can intelligently decide whether or not to trigger the packages commands in base.yml. If you don't mind, I'll start looking into that -- if you have any pointers, suggestions, or comments I'd appreciate them!

dotcarls avatar Aug 16 '20 23:08 dotcarls