amazon-linux-2023 icon indicating copy to clipboard operation
amazon-linux-2023 copied to clipboard

[Bug] - amazon-chrony-config systemd service/timer create an ordering cycle with chronyd

Open anish opened this issue 5 months ago • 3 comments

Describe the bug The chrony-config service shipped as part of amazon-chrony-config rpm is configured to be ordered both before and after chronyd at the same time

chronyd.service: Found ordering cycle on chrony-config.service/start
chronyd.service: Found dependency on chrony-config.timer/start
chronyd.service: Found dependency on chronyd.service/start
chronyd.service: Job chrony-config.service/start deleted to break ordering cycle starting with chronyd.service/start
[ SKIP ] Ordering cycle found, skipping chrony-config.service

To Reproduce Steps to reproduce the behavior:

  1. Start an instance with chronyd enabled
  2. The following error messages are seen on startup :
chronyd.service: Found ordering cycle on chrony-config.service/start
chronyd.service: Found dependency on chrony-config.timer/start
chronyd.service: Found dependency on chronyd.service/start
chronyd.service: Job chrony-config.service/start deleted to break ordering cycle starting with chronyd.service/start
[ SKIP ] Ordering cycle found, skipping chrony-config.service

This could in theory also result in the chronyd service being skipped, resulting in misconfigured instances 3. This is obvious in the systemd service file for chrony-config.service, which declares a Before dependency vs the chrony-config.timer which declares an After dependency on chronyd.service. Timer units have an implicit Before dependency on the services they activate

bash-5.2# rpm -qf /usr/lib/systemd/system/chrony-config.service 
amazon-chrony-config-4.3-1.amzn2023.0.5.noarch
bash-5.2# rpm -qf /usr/lib/systemd/system/chrony-config.timer
amazon-chrony-config-4.3-1.amzn2023.0.5.noarch

bash-5.2# cat /usr/lib/systemd/system/chrony-config.service 
[Unit]
Description=Configures optimal endpoints for the chrony NTP client in AWS
Wants=network-online.target
After=network-online.target
Before=chronyd.service

[Service]
Type=oneshot
ExecStart=/usr/sbin/amazon-chrony-config
ExecStartPost=/usr/bin/touch /run/chrony.d/.configured
RemainAfterExit=no

[Install]
WantedBy=chronyd.service

bash-5.2# cat /usr/lib/systemd/system/chrony-config.timer
[Install]
WantedBy=timers.target

[Unit]
Description=Refresh chrony time sources based on DHCP lease renewals
After=chronyd.service

[Timer]
OnActiveSec=1800
OnUnitInactiveSec=1800
RandomizedDelaySec=180

This means chrony-config.timer wants to run after chronyd.service but before chrony-config.service, and chrony-config.service wants to run before chronyd.service creating a circular depedency Expected behavior There should be no ordering cycle

Additional context Add any other context about the problem here.

anish avatar Aug 02 '25 07:08 anish

Hello @anish, thank you for raising this issue! We are looking into this and we will get back to you with an update after we investigate. Thanks!

xcawolfe-amzn avatar Aug 19 '25 17:08 xcawolfe-amzn

having issue with chronyd time getting out of sync by several minutes in some cases causing errors.

Image Image Image

CloudScubaSteve avatar Oct 21 '25 15:10 CloudScubaSteve

^-- The above screenshots are based on amazon-eks-node-al2023-x86_64-standard-1.32-v20250801 So a pretty recent release, based on multiple chrony bugs being open, it looks like updating to latest won't help.

neoakris avatar Oct 21 '25 17:10 neoakris