Several non-existent management servers

[tobzsc]

ISSUE TYPE

• Bug Report

COMPONENT NAME

UI

CLOUDSTACK VERSION

4.18.0.0
4.18.1.0

CONFIGURATION

Nothing special as this seems not to be configuration related.

OS / ENVIRONMENT

N/A

SUMMARY

After an update from 4.18.0.0 to 4.18.1.0 we noticed that we have several management servers in down state under /client/#/managementserver. We only have one management server and would expect there to be only one single server. Also see the picuture.

STEPS TO REPRODUCE

I do not have an idea how to reproduce this.

EXPECTED RESULTS

ACTUAL RESULTS

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[rajujith]

@tobzsc These are possibly due to stale entries in the mshost table, can you check:

select id,msid,name,state,version,service_ip,last_update,removed from mshost;

[tobzsc]

mysql> select id,msid,name,state,version,service_ip,last_update,removed from mshost;
+----+-----------------+----------------------+-------+----------+------------+---------------------+---------------------+
| id | msid            | name                 | state | version  | service_ip | last_update         | removed             |
+----+-----------------+----------------------+-------+----------+------------+---------------------+---------------------+
|  1 | 139438635443863 | mgmt01.cloud.asdf.com | Down  | 4.18.0.0 | 10.100.0.1 | 2023-07-21 11:53:53 | 2023-07-28 22:16:49 |
|  2 | 195765048550806 | mgmt01.cloud.asdf.com | Down  | 4.18.0.0 | 10.100.0.1 | 2023-07-24 15:58:55 | 2023-07-28 22:17:01 |
|  3 |  95390881156670 | mgmt01.cloud.asdf.com | Down  | 4.18.0.0 | 10.100.0.1 | 2023-07-27 11:01:36 | 2023-07-28 22:17:03 |
|  4 |  50929507305524 | mgmt01.cloud.asdf.com | Down  | 4.18.0.0 | 10.100.0.1 | 2023-07-31 12:42:19 | NULL                |
|  5 |  55476218781091 | mgmt01.cloud.asdf.com | Down  | 4.18.0.0 | 10.100.0.1 | 2023-08-02 13:20:40 | NULL                |
|  6 | 182968675161706 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-11-10 10:31:58 | NULL                |
|  7 | 209385682338933 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-11-13 14:03:35 | NULL                |
|  8 | 204860996050848 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-11-29 12:56:24 | NULL                |
|  9 | 279468430192113 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-11-29 15:38:39 | NULL                |
| 10 |  12007795354167 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-12-04 15:18:33 | NULL                |
| 11 |  46291004457368 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-12-04 15:28:39 | NULL                |
| 12 | 143822807052936 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-12-04 15:41:48 | NULL                |
| 13 | 187737961497700 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-12-05 10:09:17 | NULL                |
| 14 | 103509587851758 | mgmt01.cloud.asdf.com | Down  | 4.18.1.0 | 10.100.0.1 | 2023-12-05 13:39:14 | NULL                |
| 15 | 266940830264052 | mgmt01.cloud.asdf.com | Up    | 4.18.1.0 | 10.100.0.1 | 2024-01-16 12:23:36 | NULL                |
+----+-----------------+----------------------+-------+----------+------------+---------------------+---------------------+
15 rows in set (0.00 sec)

From my understanding I should set all other entries to removed except the one which is active.

[rajujith]

From my understanding I should set all other entries to removed except the one which is active.

Yes.

[tobzsc]

That fixed the issue. But still we don't know why it happened.

[DaanHoogland]

@tobzsc , I am not sure, but this might be due to breaking an operation half way, i.e. start and stop repeatedly ?

[weizhouapache]

That fixed the issue. But still we don't know why it happened.

@tobzsc is the mac address changed frequently

[tobzsc]

MAC address was never changed. We are not using DHCP.

[rohityadavcloud]

We should be able to classify this as a bug - so CloudStack shouldn't create duplicates in mshost table.

[DaanHoogland]

If we don't know why it happened and how to reproduce we can not classify it as a bug @rohityadavcloud

[weizhouapache]

@tobzsc can you share the /etc/hosts file ? if /etc/hosts is updated, the host is resolved as different IPs (127.0.0.1, or 127.0.1.1, or host IP), cloudstack will consider it as a new management server.

[tobzsc]

Of course. Here you are:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

We do not have more in it.

[weizhouapache]

Of course. Here you are:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

We do not have more in it.

Can you add a record for your server?

'ServerIP hostname'

[tobzsc]

Yes, we can add it. We will check with the next update if the problem is solved and get back to you.

[rohityadavcloud]

I think this is fixed by https://github.com/apache/cloudstack/pull/8988 and same as https://github.com/apache/cloudstack/issues/8174

Closing, pl test 4.19.1.0 when it is released in future, or try the nightlies http://download.cloudstack.org/testing/nightly/latest/

If you still get this issue, pl re-open this issue or log a new one. Thanks.