bbb-install.sh stuck during update

[cschmitt-rma]

I've used the following command to update my BigBlueButton instance for about half a year:

wget -qO- https://ubuntu.bigbluebutton.org/bbb-install-2.5.sh | bash -s -- -w -a -v focal-250 -s <our-bbb-server-fqdn> -e <my-email-adress> -c <our-turn-server-fqdn:our-token>

Today I wanted to update to the newest release, but bbb-install keeps stalling or directly after the part where it seems to run 'apt update' and 'apt dist-upgrade' (full output at the end of this issue):

Fetched 336 kB in 1s (319 kB/s) Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. ........................................................................

What I've tried so far:

1. Letting the script run for half an hour in the hope that it will continue or that some timeout will occur (neither happend)
2. Making sure that there is no principal problem with apt update/upgrade by installing a random package (worked)
3. Running 'dpkg --configure -a' and restarting the machine before running the script again (behavior unchanged)
4. Making sure that the external firewall is not obstructing some commands after update/upgrade by temporarily turning it off
5. Keeping an eye on tail -f /var/log/syslog while running the script (no messages pop up)
6. Trying to find out where the script could be stuck by reading its code (don't have the bash script knowledge understand it)

Could you please look into this, as this left my bbb instance in a non-working state (error 1004 during every echo test, probably due to the configuration part of the script not being reached)?

Here are the full outputs of bbb-install-2.5.sh and bbb-conf --check (the warnings only appeared after the failed update and bbb seems to have "forgotten" about our TURN server due to the script not finishing):

EDIT: GitHub kept garbling up the output, so here's a ControlC link: https://controlc.com/c308f4a6

[antobinary]

Hi @cschmitt-rma

Around ~3 hours ago we reverted a recent pull request https://github.com/bigbluebutton/bbb-install/pull/584 which was reported to be causing some issues when installing. The console log you attached does not bear direct resemblence to that issue but in case you were hitting your issue in the timeframe ~24h ago until 3 hours ago, I think it's worth to retry your command.

[cschmitt-rma]

@antobinary Thanks for the hint, but I ran the command during the last 15 minutes and it still produced errors.

But I was able to find where the script got stuck by adding indentifiers to all blocks of while [ <condition> ]; do sleep 1; echo -n '.'; done and running this modified version. It got stuck here ("api-demos" my change):

if [ -n "$API_DEMOS" ]; then need_pkg bbb-demo while [ ! -f /var/lib/$TOMCAT_USER/webapps/demo/bbb_api_conf.jsp ]; do sleep 1; echo -n 'api-demos.'; done fi

So I omited the "-a" parameter and now the script finished cleanly without any bbb-conf --check warnings afterwards! I had kept the "-a" in for some time now, because leaving it out kept causing tomcat9 to be uninstalled after updates.

[antobinary]

Thanks for the follow up! You definitely don't want bbb-demo in your production servers as it would allow others to join/list, etc meetings live.

About bbb-demo -- we've actually removed it from BBB 2.6+. Nothing jumps to mind right now about why this would be blocking the install completion.

[cschmitt-rma]

Thanks for the advice, I checked and the bbb-demo package is not installed, so I should be safe, right?

My issue is therefore resolved, but could this ticket maybe be counted as a request for enhancement in the sense that clarifying outputs like e.g. "Checking if port 443 is available" could be added before the three while [ <condition> ]; do sleep 1; echo -n '.'; done blocks? This would have saved me a lot of time searching for the cause of the stalling.

A very big "Thank you!" to all BBB contributors for your hard work and implementing a great video conferencing solution, happy holidays and a great 2023 to all of you!