foreman icon indicating copy to clipboard operation
foreman copied to clipboard

Fixes #35378 - Add systemd first boot service for host provisioning

Open Dyrkon opened this issue 1 year ago • 14 comments

This PR implements some improvements suggested in RFC - Systemd first boot service for host provisioning.

The main one is reducing %post section and moving some of it to service which is going to run after the first reboot of the machine. This should also ensure that the callback to foreman indicating that the build is done is going to be made only from a machine that successfully rebooted and is ready to use.

This solution was successfully tested on Centos 7, Stream 8 and Stream 9 with libvirt.

Dyrkon avatar Apr 04 '23 10:04 Dyrkon

Can one of the admins verify this patch?

theforeman-bot avatar Apr 04 '23 10:04 theforeman-bot

Can one of the admins verify this patch?

theforeman-bot avatar Apr 04 '23 10:04 theforeman-bot

Can one of the admins verify this patch?

theforeman-bot avatar Apr 04 '23 10:04 theforeman-bot

Isn't this reinventing cloud-init? Foreman already has cloud-init templates so would it make sense to leverage cloud-init? Even if it's only to call home.

I read through CloudInit default template and it seem that it would be possible to use it for the call home. I am however not sure what is the preferred way here.

If cloud-init is already being installed, then it is definitely better. If it isn't, then the service approach is at least IMO better solution.

Dyrkon avatar Apr 06 '23 09:04 Dyrkon

Thanks for the recommendation @ekohl . I have looked into cloud-init, but I am not very familiar with it, so please tell me if I got something wrong.

I have tested cloud-init by replacing the post section with this piece of code from an existing cloud-init template.

phone_home:
  url: <%= foreman_url('built') %>
  post: []
  tries: 10

The result is the same as with my solution, but there is one thing I am a bit worried about.

Cloud-init requires systemd. This is a problem as we need to still support RHEL6, which does not have a systemd. In the case of a systemd service, you can solve that by taking the script used for the callback and making it remove itself/move to a different location or rename itself. It can be then added into a crontab which will run after reboot. I am not sure how to solve this issue with cloud-init.

Dyrkon avatar Jul 10 '23 12:07 Dyrkon

I have discovered, that @reboot crontab runs the script, but the request sent by the script never arrives. A very simple solution to this problem is adding sleep before the execution of the script: @reboot sleep 60;foo.sh.

Has anyone came across this problem? I have not found solution other then migrating to systemd service, which is not an option on non-systemd systems this is supposed to run on.

Dyrkon avatar Aug 07 '23 10:08 Dyrkon

The logic deciding if service or script is used is now in separate file with specific implementation.

Dyrkon avatar Aug 21 '23 11:08 Dyrkon

Currently I am exploring the usage of initd script which works fine as the network is ready before executing any of these scripts. As systemd services replaced initd scripts I think this is the way to go.

Dyrkon avatar Aug 24 '23 11:08 Dyrkon

Should be ready to test. The current solution uses systemd and for OSes that don't support systemd (RHEL6), there is also init.d script prepared.

Dyrkon avatar Aug 29 '23 12:08 Dyrkon

ok to test

stejskalleos avatar Oct 11 '23 06:10 stejskalleos

This PR is ready for review. Does anyone have any comments or questions regarding the implementation?

Dyrkon avatar Nov 08 '23 13:11 Dyrkon

Does this affect other systems, too? Will you try to have this changes for Debian/Ubuntu/SuSE and other systems, too?

sbernhard avatar Jan 22 '24 11:01 sbernhard

Does this affect other systems, too?

Right now only the RHEL family

Will you try to have these changes for Debian/Ubuntu/SuSE and other systems, too?

We need to verify/merge it first, keep it for a while to see if it's not breaking any stuff, and later on implement it also for Debian. But no work is planned in the upcoming future.

stejskalleos avatar Jan 22 '24 11:01 stejskalleos

@Dyrkon can you please rebase the branch over the develop?

stejskalleos avatar Jan 29 '24 08:01 stejskalleos