[Question] jail startup order

[janondrusek]

Hi! I've been recently playing with Bastille and attempting a "migrate exercise" of my Docker Compose toy project. Suppose I have 3 Bastille managed jails:

1. MySQL DB exposing port 3306
2. Tomcat App server exposing ports 8080 / 8443
3. Nginx reverse proxy exposing port 80 / 443

I need to start them exactly in the order listed and ports should be available before the next jail starts. Is there a good mechanism to achieve this with Bastille?

Thanks!

[yaazkal]

One way of indicating dependencies could be with the config subcommand like this:

bastille config nginx set depend tomcat,mysql

(guessing your jails have the same name as the service they host).

This way when you start nginx it will check if tomcat and mysql are started and if not, it will start them.

So a simple nginx Bastillefile template can look like this:

PKG nginx
SYSRC nginx_enable=YES
CONFIG set depend tomcat,mysql

Of course you can set the tomcat jail to depend on the mysql jail and then set nginx to depend only on tomcat. Just letting you know that you can set depend to one or more jails.

Edit: Don't forget that the config subcommand will take effect the next time you start the jail, so restart it as your convenience.

[pmather]

I have been using iocage for a long time but recently decided to migrate to Bastille because I prefer its low-dependency requirements; Bastillefile; and so on. I exported my iocage jails and imported them into Bastille via bastille import.

I had been using jail startup dependencies successfully in iocage but find this doesn't work using the jail depend mechanism in Bastille. For example, I want to have my postgresql jail start up after my salt jail has started up because I use the latter to manage the former. This doesn't work, though:

chumby ~# bastille config postgresql set depend salt
A restart is required for the changes to be applied. See 'bastille restart postgresql'.
chumby ~# bastille restart postgresql
pfctl: /dev/pf: No such file or directory
rdr-anchor not found in pf.conf
[postgresql]:
jail: postgresql: depends on undefined jail "salt"

[postgresql]: Already started.

chumby ~#

I presume this fails because each Bastille-managed jail has its own separate jail.conf file under .../jails/<jailname> and so each jail would have no awareness of other defined jails. It's my understanding that nder FreeBSD, all jails are defined in a communal jail.conf and hence have awareness of each other when it comes to evaluating the depend keyword. The separate jail.conf for each jail means that in Bastille this awareness is not present.

A further issue arising from this is that using depend as suggested above will cause Bastille commands to fail silently. For example:

chumby ~# bastille stop postgresql
pfctl: /dev/pf: No such file or directory
rdr-anchor not found in pf.conf
[postgresql]:
jail: postgresql: depends on undefined jail "salt"

chumby ~# bastille console postgresql
[postgresql]:
Last login: Mon Mar 14 10:34:10 on pts/1
FreeBSD 13.1-PRERELEASE (CHUMBY) #1 stable/13-n249889-4340df3418e: Sat Mar  5 12:06:15 EST 2022

Chumby Database Server

If you are not authorised to use this system, log off now.

root@postgresql:~ # logout

chumby ~#

This illustrates that bastille stop postgresql fails to stop the postgresql jail, as evidenced by the ability to log into it via bastille console after the stop command has been issued. This appears to be a bug.

[joh-ku]

One workaround is to use bastille_list in /etc/rc.conf like:

bastille_list="jail1 jail2 jail3"

[pmather]

I am actually using bastille_list as a partial workaround. I say partial because it doesn't have the same semantics as dependencies have under iocage: Bastille will start and stop jails in the order given in bastille_list instead of stopping in reverse dependency order. E.g., if gitlab depends upon the postgresql jail, I need postgresql to be started before gitlab. However, for a clean shutdown, I need gitlab shut down before postgresql.

So, a complete solution ideally needs a more sophisticated dependency handling than bastille_list provides.

[cedwards]

I've created a small patch for this. Within the startup script it now reverses the order of the bastille_list before stopping the jails.

Please test this and let me know if it resolves your shutdown ordering issue. If so I'll merge it into the main app.

diff --git a/usr/local/etc/rc.d/bastille b/usr/local/etc/rc.d/bastille
index 870a285..7d16ba4 100755
--- a/usr/local/etc/rc.d/bastille
+++ b/usr/local/etc/rc.d/bastille
@@ -51,6 +51,8 @@ bastille_stop()

     local _jail

+    ## reverse order of list for shutdown
+    bastille_list=$(echo "${bastille_list}" | awk '{ for (i=NF; i>1; i--) printf("%s ",$i); print $1; }')
     for _jail in ${bastille_list}; do
         echo "Stopping Bastille Container: ${_jail}"
         ${command} stop ${_jail}

[joh-ku]

I've created a small patch for this. Within the startup script it now reverses the order of the bastille_list before stopping the jails.

Please test this and let me know if it resolves your shutdown ordering issue. If so I'll merge it into the main app.

diff --git a/usr/local/etc/rc.d/bastille b/usr/local/etc/rc.d/bastille
index 870a285..7d16ba4 100755
--- a/usr/local/etc/rc.d/bastille
+++ b/usr/local/etc/rc.d/bastille
@@ -51,6 +51,8 @@ bastille_stop()

     local _jail

+    ## reverse order of list for shutdown
+    bastille_list=$(echo "${bastille_list}" | awk '{ for (i=NF; i>1; i--) printf("%s ",$i); print $1; }')
     for _jail in ${bastille_list}; do
         echo "Stopping Bastille Container: ${_jail}"
         ${command} stop ${_jail}

@cedwards: Successfully tested, thanks!

[pmather]

Thank you. This helps a lot.

[pmather]

Actually, I spoke too soon. The start and stop methods of the Bastille rc.d script work fine with this change, but the restart method doesn't. The reversed bastille_list also gets used when doing the bastille_start phase of the restart_cmd.

[cedwards]

I will tinker with this a bit more. Looks like it needs to be set and unset (the missing component here) instead of just changed and left in reverse order.

[pmather]

I agree. In bastille_stop() you could save a copy of bastille_list to a local variable at the start of the function and then restore bastille_list from this copy at the end of the function, before exiting.

[cedwards]

Initial testing for this patch works here for start, stop and restart. Please confirm.

dependency should be "DNS server" then "WEB server" (adguard -> pkg-server) in this example.

diff --git a/usr/local/etc/rc.d/bastille b/usr/local/etc/rc.d/bastille
index 870a285..7d16ba4 100755
--- a/usr/local/etc/rc.d/bastille
+++ b/usr/local/etc/rc.d/bastille
@@ -51,6 +51,8 @@ bastille_stop()

     local _jail

+    ## reverse order of list for shutdown
+    bastille_revlist=$(echo "${bastille_list}" | awk '{ for (i=NF; i>1; i--) printf("%s ",$i); print $1; }')
+    for _jail in ${bastille_revlist}; do
         echo "Stopping Bastille Container: ${_jail}"
         ${command} stop ${_jail}

arya ~ # service bastille stop
Stopping Bastille Container: pkg-server
DEBUG: pkg-server adguard
nat cleared
[pkg-server]:
pkg-server: removed

Stopping Bastille Container: adguard
DEBUG: pkg-server adguard
nat cleared
[adguard]:
adguard: removed

arya ~ # service bastille start
Starting Bastille Container: adguard
[adguard]:
adguard: created

Starting Bastille Container: pkg-server
[pkg-server]:
pkg-server: created

arya ~ # service bastille restart
Stopping Bastille Container: pkg-server
DEBUG: pkg-server adguard
nat cleared
[pkg-server]:
pkg-server: removed

Stopping Bastille Container: adguard
DEBUG: pkg-server adguard
nat cleared
[adguard]:
adguard: removed

Starting Bastille Container: adguard
[adguard]:
adguard: created

Starting Bastille Container: pkg-server
[pkg-server]:
pkg-server: created

[pmather]

Thanks, Christer. I've tested your latest patch and it works correctly for me, too.

[tucoinfo]

A useful side effect when you don't want to maintain the list for startup.

bastille_list="mariadb ALL"

Shutdown reverse order will take some more thought.