[4.20.1] Exception when upgrade cloudstack-management

[weizhouapache]

trafficstars

problem

in fresh installation of 4.20.1-RC

2025-06-03 13:15:19,539 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_network_offering_details__name ON network_offering_details (name)
2025-06-03 13:15:19,566 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Created index i_network_offering_details__name
2025-06-03 13:15:19,569 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_network_offering_details__resource_id__resource_type ON network_off
ering_details (resource_id, resource_type)
2025-06-03 13:15:19,570 WARN  [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Unable to create index i_network_offering_details__resource_id__resource_type java.sql.SQLSyntaxErrorException: Key column 'resource_id' doesn't exist in table
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:121)
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
        at com.mysql.cj.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:555)
        at com.mysql.cj.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:339)
        at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:354)
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
        at com.cloud.upgrade.dao.DatabaseAccessObject.createIndex(DatabaseAccessObject.java:110)
        at com.cloud.upgrade.dao.DbUpgradeUtils.addIndexIfNeeded(DbUpgradeUtils.java:30)
        at com.cloud.upgrade.dao.Upgrade42000to42010.addIndexes(Upgrade42000to42010.java:103)
        at com.cloud.upgrade.dao.Upgrade42000to42010.performDataMigration(Upgrade42000to42010.java:56)
        at com.cloud.upgrade.DatabaseUpgradeChecker.executeUpgrade(DatabaseUpgradeChecker.java:367)
        at com.cloud.upgrade.DatabaseUpgradeChecker.executeUpgrades(DatabaseUpgradeChecker.java:339)
        at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:305)
        at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:478)
        at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:62)
        at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:52)
        at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
        at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
        at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
        at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
        at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:937)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:174)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:143)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:274)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:279)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:262)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:131)
        at org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
        at org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
        at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:71)
        at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:58)
        at org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.<init>(CloudStackSpringContext.java:62)
        at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.contextInitialized(CloudStackContextLoaderListener.java:53)
        at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
        at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
        at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:765)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.server.Server.start(Server.java:423)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.apache.cloudstack.ServerDaemon.start(ServerDaemon.java:204)
        at org.apache.cloudstack.ServerDaemon.main(ServerDaemon.java:115)

2025-06-03 13:15:19,578 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_service_offering__cpu ON service_offering (cpu)
2025-06-03 13:15:19,634 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Created index i_service_offering__cpu
2025-06-03 13:15:19,639 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_service_offering__speed ON service_offering (speed)
2025-06-03 13:15:19,694 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Created index i_service_offering__speed
2025-06-03 13:15:19,700 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_service_offering__ram_size ON service_offering (ram_size)
2025-06-03 13:15:19,802 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Created index i_service_offering__ram_size
2025-06-03 13:15:19,807 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_op_host_planner_reservation__resource_usage ON op_host_planner_reservation (resource_usage)

versions

The versions of ACS, hypervisors, storage, network etc..

The steps to reproduce the bug

...

What to do about it?

No response

[weizhouapache]

error :

2025-06-03 13:15:19,569 DEBUG [c.c.u.d.DatabaseAccessObject] (main:[]) (logid:) Statement: CREATE INDEX i_network_offering_details__resource_id__resource_type ON network_off ering_details (resource_id, resource_type)

[DaanHoogland]

@weizhouapache , this sounds like a blocker for 4.20.1. Is it?

[weizhouapache]

@DaanHoogland Not really. The SQL upgrade continued

[bernardodemarco]

The following line of code seems to be the root cause of the exception:

https://github.com/apache/cloudstack/blob/8e4fe1c370e251656f3314bc9f522f6491be0e98/engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade42000to42010.java#L103

The cloud.network_offering_details table does not have the resource_id and resource_type columns. @weizhouapache @DaanHoogland @shwstppr, should we simply remove that line?

[shwstppr]

@bernardodemarco done. PR: #11171

[bernardodemarco]

@bernardodemarco done. PR: https://github.com/apache/cloudstack/pull/11171

@shwstppr, perfect, thanks!

[DaanHoogland]

fixed in #11171