Systemd and volumio files not properly updated

[volumio]

@ashthespy

Issue:

• The custom files in https://github.com/volumio/Build/tree/volumioOS/volumio are copied over too soon.
• When installing custom packages https://github.com/volumio/Build/blob/volumioOS/build.sh#L548 for example, the systemd file in the custom package overwrites the volumio custom one

GOAL:

• Have the files https://github.com/volumio/Build/tree/volumioOS/volumio copied as late as possible in the build process, following the pattern used in https://github.com/volumio/Build/blob/master/scripts/configure.sh
• This will make sure that all the customization we've done on those files are respected
• Please send a PR (tested) to volumioOS

[ashthespy]

This is something I wanted to clarify -- this part is from the old build system that I copied over, but it would be good to refactor.

Rootfs Creation

-- Run multistrap (volumioOS master) -- Run volumioconfig.sh in chroot (volumioOS master) -- Run configure.sh on the resulting rootfs (volumioOS master)

Image creation

-- Run <device>image.sh (makeimage.sh in volumioOS --- Run <device>config.sh in chroot. (chrootconfig.sh in volumioOS) --- Run finalize.sh(volumioOS master)

The structure here is a bit tricky, as volumioconfig seems to have initially grown from davidmonro/pi_builder/configscript.sh (or the other way around?) -- but the purpose is mainly to finish the multistrap Debconf and pre-seeding. But as volumioconfig is currently, it is doing some sketchy things like symlinking non existent files -- the service files are copied over only later in configure.sh. It also looks like some tried enabling the service, and then found a workaround.. ;-) https://github.com/volumio/Build/blob/e81b2037a65d8f554cb0c857a1007d659ad7f838/scripts/volumioconfig.sh#L461-L463

Which leads to the question(s)

-- Should volumioconfig be trimmed back to it's original purpose -- finish Debconf configuration after multistrap?

-- When should which configuration be actually done?

--- Some configuration files are created in volumioconfig.sh https://github.com/volumio/Build/blob/7e70a9a70ccd64d4428d7e4fd88b97d5d2d1a8ba/scripts/volumio/volumioconfig.sh#L361-L366 --- Some files from /volumio are copied over in configure.sh (such as alsa-base.conf) https://github.com/volumio/Build/blob/7e70a9a70ccd64d4428d7e4fd88b97d5d2d1a8ba/scripts/volumio/configure.sh#L40-L41

tl;dr --- We should consider refactoring this part of the build system as well - I only worked on the modularity of the device creation, but it would be good to go consider what is actually required, and how to do it.

That being said, I am a bit preoccupied with life until end of March, so might be good to find a short term solution if this is a blocker.

[volumio]

@ashthespy IMHO, given the lack of time and the absolute priority (which is sending a testable build to the community ASAP) I suggest:

• Working on the immediate issue, let's keep refactors for later
• Originally, custom packages were installed in volumioconfig and THEN configure would place the correct files in the system (including systemd ) https://github.com/volumio/Build/blob/master/build.sh#L209
• Let's just add "configure" after the install of custom packages, or move the install of custom packages before configure (or whenever the files are copied)

That will unlock the situation and allow to start testing ;) After that your ideas of refactoring can be good.

[ashthespy]

• Originally, custom packages were installed in volumioconfig and THEN configure would place the correct files in the system (including systemd )
• Let's just add "configure" after the install of custom packages, or move the install of custom packages before configure (or whenever the files are copied)

Yep, you could still do that - have a look at how the configuration variables are passed into the chroot scripts and packages are installed chrootconfig.sh you could copy that over to volumioconfig so that is done at a rootfs level, rather than at the device level.

Thus far the assumption was rootfs == all packages available from a proper package repository. So dependencies would be automatically managed, and apt would have a completely populated list of packages installed so we have a predictable base rootfs that we can modify as required in the device level scripts.

You could also go back to installing things in a simpler fashion as previously https://github.com/volumio/Build/blob/e81b2037a65d8f554cb0c857a1007d659ad7f838/scripts/volumioconfig.sh#L189-L219

[volumio]

Thanks for the hint. What I still don't understand is when the custom systemd files are copied. Could you point me to that please?

[ashthespy]

That part remains unchanged - it will be the same as in the old system so should be somewhere in configure.sh

EDIT: Here you go https://github.com/volumio/Build/blob/e81b2037a65d8f554cb0c857a1007d659ad7f838/scripts/configure.sh#L60-L61

Minimal effort way to fix this for now if you ask me - is to add that part to finalize.sh instead...

[volumio]

@ashthespy I tried with https://github.com/volumio/Build/commit/23d70acea9c27e2819838a3be7f0378d726c0c2a

but packages do not get installed... Sorry if I ask, but I need pointers here since I still struggle to get the chronology of what is called when. When would you place the custom package install so it gets installed before configure?

[ashthespy]

Hmm with 23d70ac you would break the KIOSKMODE which is a device level configuration. I don't believe you want ALL rootfs's created to contain all the kiosk dependencies by default?

When would you place the custom package install so it gets installed before configure?

-- For that snippet to work, /volumio/customPkgs needs to contain the correct packages. So either they would have to be extracted into rootfs either by multistrap directly, or fetched from the repo in volumioconfig as previously, or copied into the rootfs manually before volumioconfig.sh is run..

[ashthespy]

Issues after switching to Volumio builds of shairport and upmpdcli are fixed with https://github.com/volumio/Build/commit/e446a7f1b7b570d35771f25d01639d8fc4166f4a

Leaving this open for future refactoring..