Only one `mon` host in ceph.conf on multi node cluster

[sabaini]

Setting up a 3-node cluster yet the ceph.conf does not get updated with initial mon host entry. I would have expected all 3 MONs of the cluster showing up .

This happens when setting up a bootstrap node first and then add-unit 2 more nodes -- when setting up all 3 nodes at once this appears not to happen.

$ juju deploy microceph --constraints="virt-type=virtual-machine root-disk=12G mem=4G" --base [email protected] --config snap-channel=squid/edge
Deployed "microceph" from charm-hub charm "microceph", revision 112 in channel squid/stable on [email protected]/stable

$ juju status
Model    Controller  Cloud/Region         Version  SLA          Timestamp
default  lxd         localhost/localhost  3.6.5    unsupported  20:21:47Z

App        Version  Status  Scale  Charm      Channel       Rev  Exposed  Message
microceph           active      1  microceph  squid/stable  112  no       

Unit          Workload  Agent  Machine  Public address                         Ports  Message
microceph/0*  active    idle   0        fd42:73df:d6b9:5e4:216:3eff:fe9a:f078         

Machine  State    Address                                Inst id        Base          AZ  Message
0        started  fd42:73df:d6b9:5e4:216:3eff:fe9a:f078  juju-a3dcaa-0  [email protected]      Running

$ juju ssh 0 -- sudo grep mon /var/snap/microceph/current/conf/ceph.conf
mon host = 10.4.251.3

$ juju add-unit -n2 microceph

$ juju status
Model    Controller  Cloud/Region         Version  SLA          Timestamp
default  lxd         localhost/localhost  3.6.5    unsupported  20:26:09Z

App        Version  Status  Scale  Charm      Channel       Rev  Exposed  Message
microceph           active      3  microceph  squid/stable  112  no       

Unit          Workload  Agent  Machine  Public address                         Ports  Message
microceph/0*  active    idle   0        fd42:73df:d6b9:5e4:216:3eff:fe9a:f078         
microceph/1   active    idle   1        10.4.251.40                                   
microceph/2   active    idle   2        fd42:73df:d6b9:5e4:216:3eff:fe17:7a06         

Machine  State    Address                                Inst id        Base          AZ  Message
0        started  fd42:73df:d6b9:5e4:216:3eff:fe9a:f078  juju-a3dcaa-0  [email protected]      Running
1        started  10.4.251.40                            juju-a3dcaa-1  [email protected]      Running
2        started  fd42:73df:d6b9:5e4:216:3eff:fe17:7a06  juju-a3dcaa-2  [email protected]      Running


$ juju ssh 0 -- sudo grep mon /var/snap/microceph/current/conf/ceph.conf
mon host = 10.4.251.3

$ juju ssh 0 -- sudo ceph mon stat
e4: 3 mons at {juju-a3dcaa-0=[v2:10.4.251.3:3300/0,v1:10.4.251.3:6789/0],juju-a3dcaa-1=[v2:10.4.251.40:3300/0,v1:10.4.251.40:6789/0],juju-a3dcaa-2=[v2:10.4.251.181:3300/0,v1:10.4.251.181:6789/0]} removed_ranks: {} disallowed_leaders: {}, election epoch 14, leader 0 juju-a3dcaa-0, quorum 0,1,2 juju-a3dcaa-0,juju-a3dcaa-1,juju-a3dcaa-2

$ juju ssh 0 -- snap list
Name       Version                Rev    Tracking       Publisher   Notes
core24     20250318               888    latest/stable  canonical✓  base
microceph  19.2.0+snap84b63149e5  1381   squid/edge     canonical✓  held
snapd      2.68.4                 24505  latest/stable  canonical✓  snapd

[syncronize-issues-to-jira[bot]]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CEPH-1306.

This message was autogenerated