[BUG] PKG Install method fails during setup, version 0.10.20231125 @ 14.0-RELEASE-p3

[em-winterschon]

[MANDATORY] Describe the bug [MANDATORY] Setup stage configure_zfs fails when following the installation process from the docs, when using pkg method.

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)

bastille -v && freebsd-version -kru
0.10.20231125
14.0-RELEASE-p3
14.0-RELEASE-p3
14.0-RELEASE-p3

[MANDATORY] How did you install bastille? (port/pkg/git) pkg

[optional] Steps to reproduce? Deploy standard ZFS install of 14.0 Release-p3, no extra configuration. Command sequence fails regardless of running as root or admin user with full sudo privs.

» pkg install bastille
Updating FreeBSD-Latest repository catalogue...
FreeBSD-Latest repository is up to date.
All repositories are up to date.
Updating database digests format: 100%
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        bastille: 0.10.20231125

Number of packages to be installed: 1

43 KiB to be downloaded.

Proceed with this action? [y/N]: y
[1/1] Fetching bastille-0.10.20231125.pkg: 100%   43 KiB  43.9kB/s    00:01
Checking integrity... done (0 conflicting)
[1/1] Installing bastille-0.10.20231125...
[1/1] Extracting bastille-0.10.20231125: 100%

» pkg info bastille
bastille-0.10.20231125
Name           : bastille
Version        : 0.10.20231125
Installed on   : Fri Dec 22 10:54:34 2023 PST
Origin         : sysutils/bastille
Architecture   : FreeBSD:14:*
Prefix         : /usr/local
Categories     : sysutils
Licenses       : BSD3CLAUSE
Maintainer     : [email protected]
WWW            : https://bastillebsd.org
Comment        : Jail automation framework
Annotations    :
        build_timestamp: 2023-12-14T22:17:29+0000
        built_by       : poudriere-git-3.3.0-1258-gbc38e2e6
        port_checkout_unclean: no
        port_git_hash  : 08ed7d0165
        ports_top_checkout_unclean: no
        ports_top_git_hash: bb504ccec5
        repo_type      : binary
        repository     : FreeBSD-Latest
Flat size      : 272KiB
Description    :
Bastille helps you quickly create and manage FreeBSD Jails.

The project focuses on secure defaults, automation and
repeatable templates.

» sysrc bastille_enable="YES"
bastille_enable:  -> YES

» sysrc bastille_rcorder=YES
bastille_rcorder:  -> YES

» bastille setup
bastille_enable: YES -> YES
Configuring bastille0 loopback interface
cloned_interfaces:  -> lo1
ifconfig_lo1_name:  -> bastille0
Bringing up new interface: bastille0
Created clone interfaces: lo1.
Determined default network interface: (bxe0)
/etc/pf.conf does not exist: creating...
pf_enable: NO -> YES
/usr/sbin/sysrc: cannot create /usr/local/bastille/bastille.conf: No such file or directory
/usr/sbin/sysrc: cannot create /usr/local/bastille/bastille.conf: No such file or directory

[optional] Expected behavior Running setup without args should complete without errors. The default config specifies bastille_prefix="/usr/local/bastille" but the directory never gets created and the default config doesn't get copied to that base dir, so the sysrc command fails when attempting to parse it during setup's configure_zfs function.

At some point before configure_zfs runs the sample config is copied to /usr/local/etc/bastille/bastille.conf and that file is parsed by setup.sh at line 32: . /usr/local/etc/bastille/bastille.conf

[optional] Screenshots N/A

[optional] Additional context

[em-winterschon]

Three commands to resolve. I can fork and generate a PR if this is reproducible for others.

» mkdir /usr/local/bastille
» chmod 0750 /usr/local/bastille
» ln -s /usr/local/etc/bastille/bastille.conf /usr/local/bastille/bastille.conf

» bastille setup
bastille_enable: YES -> YES
Configuring bastille0 loopback interface
cloned_interfaces: lo1 -> lo1
ifconfig_lo1_name: bastille0 -> bastille0
Bringing up new interface: bastille0
ifconfig: interface lo1 already exists
Determined default network interface: (bxe0)
/etc/pf.conf does not exist: creating...
pf_enable: YES -> YES
bastille_zfs_enable:  -> YES
bastille_zfs_zpool:  -> zroot

[cedwards]

I see the issue here. Lines 100,101 in setup.sh are missing the /etc/ between the prefix and the conf. It's trying to update the config in a path that, as you discovered, does not exist.

I'll try to get this patched tonight. Thank you for pointing it out.

[em-winterschon]

Some additional commands were necessary in order to get bastille bootstrap functional after a new run of setup.sh.

The prior commands related to creating a directory and setting perms were modified in favor of moving those to the zfs setup stage, as the (missing) zfs create statement must be run before anything productive can occur in the jail bootstrap phases.

I haven't looked into your releng/ci-cd/automated deploy testing process, but maybe something changed lately that should be flagging a failure state during post-pkg install.

Bootstrap Script

--- /home/eva/Projects/github/bastille/usr/local/share/bastille/bootstrap.sh    2023-12-22 12:53:44.884051000 -0800
+++ bootstrap.sh        2023-12-24 04:28:49.337620000 -0800
@@ -212,6 +212,11 @@
 }

 bootstrap_release() {
+    # Make sure to check/bootstrap directories first.¬
+    NOCACHEDIR=1
+    RELEASE="${DIR_BOOTSTRAP}"
+    bootstrap_directories
+
     ## if release exists quit, else bootstrap additional distfiles
     if [ -f "${bastille_releasesdir}/${RELEASE}/COPYRIGHT" ]; then
         ## check distfiles list and skip existing cached files
@@ -346,7 +351,7 @@
                 ;;
             esac
         else
-            # If already set in /boot/loader.conf, check and try to load the module.
+            # If already set in /boot/loader.conf, check and try to load the module.
             if ! kldstat -m ${_req_kmod} >/dev/null 2>&1; then
                 info "Loading kernel module: ${_req_kmod}"
                 kldload -v ${_req_kmod}

ZFS Setup Script

--- /home/eva/Projects/github/bastille/usr/local/share/bastille/setup.sh        2023-12-22 12:53:44.888116000 -0800
+++ setup.sh    2023-12-24 04:37:29.571119000 -0800
@@ -28,8 +28,9 @@
 # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+bastille_config="/usr/local/etc/bastille/bastille.conf"
 . /usr/local/share/bastille/common.sh
-. /usr/local/etc/bastille/bastille.conf
+. ${bastille_config}

 usage() {
     error_exit "Usage: bastille setup [pf|bastille0|zfs|vnet]"
@@ -78,8 +79,8 @@
     local ext_if
     ext_if=$(netstat -rn | awk '/default/ {print $4}' | head -n1)
     info "Determined default network interface: ($ext_if)"
-    info "${bastille_pf_conf} does not exist: creating..."
-
+    info "${bastille_pf_conf} does not exist: creating..."
+
     ## creating pf.conf
     cat << EOF > ${bastille_pf_conf}
 ## generated by bastille setup
@@ -111,8 +112,11 @@
     else
         ## attempt to determine bastille_zroot from `zpool list`
         bastille_zroot=$(zpool list | grep -v NAME | awk '{print $1}')
-        sysrc -f "${bastille_prefix}/bastille.conf" bastille_zfs_enable=YES
-        sysrc -f "${bastille_prefix}/bastille.conf" bastille_zfs_zpool="${bastille_zroot}"
+        zfs create ${bastille_zfs_options} -o mountpoint=${bastille_prefix} ${bastille_zroot}/bastille
+        chown root:root ${bastille_prefix}
+        chmod 0750 ${bastille_prefix}
+        sysrc -f "${bastille_config}" bastille_zfs_enable=YES
+        sysrc -f "${bastille_config}" bastille_zfs_zpool="${bastille_zroot}"
     fi
 }

Pre-Patch Failure Output

Prior to the two diff patches being applied, here's an example failure state from a fresh install.

¡root!» bastille bootstrap 14.0-RELEASE amd64
Bootstrapping FreeBSD distfiles...
cannot create 'zroot/bastille/backups': parent does not exist
chmod: /usr/local/bastille/backups: No such file or directory
cannot create 'zroot/bastille/cache': parent does not exist
cannot create 'zroot/bastille/cache/14.0-RELEASE': parent does not exist
cannot create 'zroot/bastille/jails': parent does not exist
cannot create 'zroot/bastille/logs': parent does not exist
cannot create 'zroot/bastille/templates': parent does not exist
cannot create 'zroot/bastille/releases': parent does not exist
cannot create 'zroot/bastille/releases/14.0-RELEASE': parent does not exist
cannot create 'zroot/bastille/backups': parent does not exist
chmod: /usr/local/bastille/backups: No such file or directory
cannot create 'zroot/bastille/cache': parent does not exist
cannot create 'zroot/bastille/jails': parent does not exist
cannot create 'zroot/bastille/logs': parent does not exist
cannot create 'zroot/bastille/templates': parent does not exist
cannot create 'zroot/bastille/releases': parent does not exist
cannot create 'zroot/bastille/releases/': trailing slash in name
fetch: /usr/local/bastille/cache//MANIFEST: open(): No such file or directory
ls: /usr/local/bastille/cache/: No such file or directory
cannot open 'zroot/bastille/cache/': trailing slash in name
ls: /usr/local/bastille/releases/: No such file or directory
cannot open 'zroot/bastille/releases/': trailing slash in name
Bootstrap failed.

Post-Patch Example Output

Applied the patches with no errors, results look good now.

» bastille bootstrap 14.0-RELEASE amd64
Bootstrapping FreeBSD distfiles...
/usr/local/bastille/cache//MANIFEST                   1046  B   22 MBps    00s
/usr/local/bastille/cache//base.txz                    190 MB   34 MBps    06s
Validated checksum for : base.txz
MANIFEST: 05a7c268aa4bdc5eb178f4611875acaaee5b210b7b57ecc2445345ac839b7cb8
DOWNLOAD: 05a7c268aa4bdc5eb178f4611875acaaee5b210b7b57ecc2445345ac839b7cb8
Extracting FreeBSD  base.txz.

Bootstrap successful.
See 'bastille --help' for available commands.