cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

Restarting management server causes multiple entries for the same node

Open rohityadavcloud opened this issue 2 years ago • 6 comments
trafficstars

Restart management server several times, even if there's only one mgmt server, it causes several entries in the cloud.mshost table and in the API:

Screenshot 2023-11-01 at 11 17 34 PM
ISSUE TYPE
  • Bug Report
COMPONENT NAME
Mgmt server
CLOUDSTACK VERSION
4.18.1.0
CONFIGURATION

Adv zone KVM env with Ubuntu 22.04

rohityadavcloud avatar Nov 01 '23 17:11 rohityadavcloud

Tried to reproduce on ubuntu22 and centos7 on 4.18-head, no go. I'll give it one more try with stock 4.18.1 on ubuntu22

DaanHoogland avatar Nov 02 '23 15:11 DaanHoogland

@rohityadavcloud can you share the entries in database table mshost ?

weizhouapache avatar Nov 02 '23 15:11 weizhouapache

If a new record each time when management server is restarted, it might be because the management server id is calculated from a random mac address.

https://github.com/apache/cloudstack/blob/78213da82860761e174d5876afe857977f3889b3/utils/src/main/java/com/cloud/utils/net/MacAddress.java#L145

may need to verify all the commands before the line in same class

weizhouapache avatar Nov 03 '23 07:11 weizhouapache

looks same as #8516

weizhouapache avatar Feb 12 '24 17:02 weizhouapache

If a new record each time when management server is restarted, it might be because the management server id is calculated from a random mac address.

https://github.com/apache/cloudstack/blob/78213da82860761e174d5876afe857977f3889b3/utils/src/main/java/com/cloud/utils/net/MacAddress.java#L145

may need to verify all the commands before the line in same class

What is the use case for this logic? Why not just use the servers FQDN?

poddm avatar Feb 16 '24 18:02 poddm

If a new record each time when management server is restarted, it might be because the management server id is calculated from a random mac address. https://github.com/apache/cloudstack/blob/78213da82860761e174d5876afe857977f3889b3/utils/src/main/java/com/cloud/utils/net/MacAddress.java#L145

may need to verify all the commands before the line in same class

What is the use case for this logic? Why not just use the servers FQDN?

@poddm Actually it does. It gets the local address at first. If it does not work, then fall back to the process above. If the issue still exists, please share more information like

  • hostname
  • hostname -f
  • /etc/hosts

weizhouapache avatar Feb 16 '24 18:02 weizhouapache

This happens on a server which is used both as management server and KVM hosts (and docker host), the number of interfaces keep on changing over time and the current code is flaky to be deterministic about which interface to pick. I've addressed this at: https://github.com/apache/cloudstack/pull/8968/commits/fb82e352bf4cd9306421effdd91bdc6e9387df5f

The solution to the issue is to only pick non-virtual, possibly actual physical Nic (last in the list seems to be correct) and use it to convert/use as the (long) ID of the mgmt server. I'm also thinking this value should be configurable via server.properties.

rohityadavcloud avatar Apr 25 '24 04:04 rohityadavcloud

Fixed by https://github.com/apache/cloudstack/pull/8988

rohityadavcloud avatar Apr 30 '24 11:04 rohityadavcloud