cloudstack
cloudstack copied to clipboard
Published
20 hours ago •
apache
apache
Unable to delete User Data
ISSUE TYPE
Bug Report
COMPONENT NAME
User Data
CLOUDSTACK VERSION
4.19
CONFIGURATION
N/A
OS / ENVIRONMENT
Oracle Linux Server 8.9 / KVM
SUMMARY
Unable to delete created User Data after registering with a Template and creating Instance from that template. After deleting the Instance and then the Template that are using User Data, unable to delete that specific user data. Attached screenshot error for reference.
STEPS TO REPRODUCE
Create a User Data. Register a Template with this user data.
Create a Instance from the template.
Delete both Instance and Template.
Try deleting User Data
EXPECTED RESULTS
User data is deleted.
ACTUAL RESULTS
User data not deleted with below error.
Thanks for opening your first issue here! Be sure to follow the issue template!
![boring-cyborg[bot] avatar](https://avatars.githubusercontent.com/in/50386?v=4 "Published by a pub.dev verified publisher"){kind=small}
Below is error log from management-server.log
2024-02-26 04:52:07,978 DEBUG [c.c.u.d.T.Transaction] (qtp139928643-20033:ctx-38ba6222 ctx-f48be2d1) (logid:e2eac635) Rolling back the transaction: Time = 0 Name = qtp139928643-20033; called by -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-TransactionLegacy.close:656-TransactionContextInterceptor.invoke:36-ReflectiveMethodInvocation.proceed:175-ExposeInvocationInterceptor.invoke:97-ReflectiveMethodInvocation.proceed:186-JdkDynamicAopProxy.invoke:215-$Proxy268.remove:-1-ManagementServerImpl.deleteUserData:4700-NativeMethodAccessorImpl.invoke0:-2-NativeMethodAccessorImpl.invoke:62
2024-02-26 04:52:07,978 ERROR [c.c.a.ApiServer] (qtp139928643-20033:ctx-38ba6222 ctx-f48be2d1) (logid:e2eac635) unhandled exception executing api command: [Ljava.lang.String;@610c1627
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: DELETE FROM user_data WHERE user_data.id= 1
at com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1207)
at com.cloud.utils.db.GenericDaoBase.remove(GenericDaoBase.java:1899)
at jdk.internal.reflect.GeneratedMethodAccessor377.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy268.remove(Unknown Source)
at com.cloud.server.ManagementServerImpl.deleteUserData(ManagementServerImpl.java:4700)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy227.deleteUserData(Unknown Source)
at org.apache.cloudstack.api.command.user.userdata.DeleteUserDataCmd.execute(DeleteUserDataCmd.java:90)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:172)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:782)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:603)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:347)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:154)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:151)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:105)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:645)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:887)
Caused by: java.sql.SQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`cloud`.`vm_template`, CONSTRAINT `fk_vm_template__user_data_id` FOREIGN KEY (`user_data_id`) REFERENCES `user_data` (`id`))
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:117)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
... 85 more
2024-02-26 04:52:07,979 DEBUG [c.c.a.ApiServlet] (qtp139928643-20033:ctx-38ba6222 ctx-f48be2d1) (logid:e2eac635) ===END=== 10.119.206.194 -- GET id=b4926eb5-d4db-48b1-b86a-879c517322b5&account=admin&domainid=849cfd03-d21c-11ee-84ad-246e96679ca0&command=deleteUserData&response=json
thanks @adidiborg i am also able to reproduce the issue with the same exception
cc @harikrishna-patnala
Caused by: java.sql.SQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`cloud`.`vm_template`, CONSTRAINT `fk_vm_template__user_data_id` FOREIGN KEY (`user_data_id`) REFERENCES `user_data` (`id`))
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:117)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
... 81 more
@kiranchavala ok, thanks for checking. We can delete User data by just creating and then deleting (i.e without attaching to template / instance). But once attached, get this error.
@kiranchavala @adidiborg can you share the result of mysql command ?
show create table vm_template;
@weizhouapache
mysql> show create table vm_template \G;
*************************** 1. row ***************************
Table: vm_template
Create Table: CREATE TABLE `vm_template` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`unique_name` varchar(255) NOT NULL,
`name` varchar(255) NOT NULL,
`uuid` varchar(40) DEFAULT NULL,
`public` int unsigned NOT NULL,
`featured` int unsigned NOT NULL,
`type` varchar(32) DEFAULT NULL,
`hvm` int unsigned NOT NULL COMMENT 'requires HVM',
`bits` int unsigned NOT NULL COMMENT '32 bit or 64 bit',
`url` varchar(255) DEFAULT NULL COMMENT 'the url where the template exists externally',
`format` varchar(32) NOT NULL COMMENT 'format for the template',
`created` datetime NOT NULL COMMENT 'Date created',
`removed` datetime DEFAULT NULL COMMENT 'Date removed if not null',
`account_id` bigint unsigned NOT NULL COMMENT 'id of the account that created this template',
`checksum` varchar(255) DEFAULT NULL COMMENT 'checksum for the template root disk',
`display_text` varchar(4096) DEFAULT NULL COMMENT 'Description text set by the admin for display purpose only',
`enable_password` int unsigned NOT NULL DEFAULT '1' COMMENT 'true if this template supports password reset',
`enable_sshkey` int unsigned NOT NULL DEFAULT '0' COMMENT 'true if this template supports sshkey reset',
`guest_os_id` bigint unsigned NOT NULL COMMENT 'the OS of the template',
`bootable` int unsigned NOT NULL DEFAULT '1' COMMENT 'true if this template represents a bootable ISO',
`prepopulate` int unsigned NOT NULL DEFAULT '0' COMMENT 'prepopulate this template to primary storage',
`cross_zones` int unsigned NOT NULL DEFAULT '0' COMMENT 'Make this template available in all zones',
`extractable` int unsigned NOT NULL DEFAULT '0' COMMENT 'Is this template extractable',
`hypervisor_type` varchar(32) DEFAULT NULL COMMENT 'hypervisor that the template belongs to',
`source_template_id` bigint unsigned DEFAULT NULL COMMENT 'Id of the original template, if this template is created from snapshot',
`template_tag` varchar(255) DEFAULT NULL COMMENT 'template tag',
`sort_key` int NOT NULL DEFAULT '0' COMMENT 'sort key used for customising sort method',
`size` bigint unsigned DEFAULT NULL,
`state` varchar(255) DEFAULT NULL,
`update_count` bigint unsigned DEFAULT NULL,
`updated` datetime DEFAULT NULL,
`dynamically_scalable` tinyint unsigned NOT NULL DEFAULT '0' COMMENT 'true if template contains XS/VMWare tools inorder to support dynamic scaling of VM cpu/memory',
`parent_template_id` bigint unsigned DEFAULT NULL COMMENT 'If datadisk template, then id of the root template this template belongs to',
`direct_download` tinyint(1) DEFAULT '0' COMMENT 'Indicates if Secondary Storage is bypassed and template is downloaded to Primary Storage',
`deploy_as_is` tinyint(1) NOT NULL DEFAULT '0' COMMENT 'True if the template should be deployed with disks and networks as defined by OVF',
`user_data_id` bigint unsigned DEFAULT NULL COMMENT 'id of the user data',
`user_data_link_policy` varchar(255) DEFAULT NULL COMMENT 'user data link policy with template',
PRIMARY KEY (`id`),
UNIQUE KEY `uc_vm_template__uuid` (`uuid`),
KEY `i_vm_template__removed` (`removed`),
KEY `i_vm_template__public` (`public`),
KEY `fk_vm_template__user_data_id` (`user_data_id`),
CONSTRAINT `fk_vm_template__user_data_id` FOREIGN KEY (`user_data_id`) REFERENCES `user_data` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=211 DEFAULT CHARSET=utf8mb3
@weizhouapache
mysql> show create table vm_template \G; *************************** 1. row *************************** Table: vm_template Create Table: CREATE TABLE `vm_template` ( `id` bigint unsigned NOT NULL AUTO_INCREMENT, `unique_name` varchar(255) NOT NULL, `name` varchar(255) NOT NULL, `uuid` varchar(40) DEFAULT NULL, `public` int unsigned NOT NULL, `featured` int unsigned NOT NULL, `type` varchar(32) DEFAULT NULL, `hvm` int unsigned NOT NULL COMMENT 'requires HVM', `bits` int unsigned NOT NULL COMMENT '32 bit or 64 bit', `url` varchar(255) DEFAULT NULL COMMENT 'the url where the template exists externally', `format` varchar(32) NOT NULL COMMENT 'format for the template', `created` datetime NOT NULL COMMENT 'Date created', `removed` datetime DEFAULT NULL COMMENT 'Date removed if not null', `account_id` bigint unsigned NOT NULL COMMENT 'id of the account that created this template', `checksum` varchar(255) DEFAULT NULL COMMENT 'checksum for the template root disk', `display_text` varchar(4096) DEFAULT NULL COMMENT 'Description text set by the admin for display purpose only', `enable_password` int unsigned NOT NULL DEFAULT '1' COMMENT 'true if this template supports password reset', `enable_sshkey` int unsigned NOT NULL DEFAULT '0' COMMENT 'true if this template supports sshkey reset', `guest_os_id` bigint unsigned NOT NULL COMMENT 'the OS of the template', `bootable` int unsigned NOT NULL DEFAULT '1' COMMENT 'true if this template represents a bootable ISO', `prepopulate` int unsigned NOT NULL DEFAULT '0' COMMENT 'prepopulate this template to primary storage', `cross_zones` int unsigned NOT NULL DEFAULT '0' COMMENT 'Make this template available in all zones', `extractable` int unsigned NOT NULL DEFAULT '0' COMMENT 'Is this template extractable', `hypervisor_type` varchar(32) DEFAULT NULL COMMENT 'hypervisor that the template belongs to', `source_template_id` bigint unsigned DEFAULT NULL COMMENT 'Id of the original template, if this template is created from snapshot', `template_tag` varchar(255) DEFAULT NULL COMMENT 'template tag', `sort_key` int NOT NULL DEFAULT '0' COMMENT 'sort key used for customising sort method', `size` bigint unsigned DEFAULT NULL, `state` varchar(255) DEFAULT NULL, `update_count` bigint unsigned DEFAULT NULL, `updated` datetime DEFAULT NULL, `dynamically_scalable` tinyint unsigned NOT NULL DEFAULT '0' COMMENT 'true if template contains XS/VMWare tools inorder to support dynamic scaling of VM cpu/memory', `parent_template_id` bigint unsigned DEFAULT NULL COMMENT 'If datadisk template, then id of the root template this template belongs to', `direct_download` tinyint(1) DEFAULT '0' COMMENT 'Indicates if Secondary Storage is bypassed and template is downloaded to Primary Storage', `deploy_as_is` tinyint(1) NOT NULL DEFAULT '0' COMMENT 'True if the template should be deployed with disks and networks as defined by OVF', `user_data_id` bigint unsigned DEFAULT NULL COMMENT 'id of the user data', `user_data_link_policy` varchar(255) DEFAULT NULL COMMENT 'user data link policy with template', PRIMARY KEY (`id`), UNIQUE KEY `uc_vm_template__uuid` (`uuid`), KEY `i_vm_template__removed` (`removed`), KEY `i_vm_template__public` (`public`), KEY `fk_vm_template__user_data_id` (`user_data_id`), CONSTRAINT `fk_vm_template__user_data_id` FOREIGN KEY (`user_data_id`) REFERENCES `user_data` (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=211 DEFAULT CHARSET=utf8mb3
thanks @kiranchavala
It looks like relates to the non-existing removed column in user_data.
Once userdata is removed, the record is gone from user_data table.
cc @harikrishna-patnala
oh, and as user-data is a first class citizen now we should have a column like that. @adidiborg adding that column could solve your issue, Did you try that?
oh, and as user-data is a first class citizen now we should have a column like that. @adidiborg adding that column could solve your issue, Did you try that?
VO needs to be updated as well
agreed to the solution to have removed column instead of removing the entry itself. I'll create a PR for that.
oh, and as user-data is a first class citizen now we should have a column like that. @adidiborg adding that column could solve your issue, Did you try that?
@DaanHoogland / @weizhouapache , no I haven't. I'm new to using Cloudstack and haven't touched its mysql DB entries yet. Can you please share SQL commands that would allow to delete User data? Thanks
agreed to the solution to have removed column instead of removing the entry itself. I'll create a PR for that.
Also, I think, it's better to have a background thread / any API to purge these removed records later (some point of time after removed, can be controlled through global setting).
agreed to the solution to have removed column instead of removing the entry itself. I'll create a PR for that.
@harikrishna-patnala an alternative solution without db changes is
- set user_data_id to null for the vm_template
- then remove user_data record (if
removedcolumn does not exist, records are expunged)
this looks very simple, and no need to clean up userdata. @sureshanaparti
oh, and as user-data is a first class citizen now we should have a column like that. @adidiborg adding that column could solve your issue, Did you try that?
@DaanHoogland / @weizhouapache , no I haven't. I'm new to using Cloudstack and haven't touched its mysql DB entries yet. Can you please share SQL commands that would allow to delete User data? Thanks
@adidiborg Follow the steps listed (assuming user data id = 1, based on the screenshot added)
- Check/Confirm there are no templates with the user data - empty result for the sql below.
SELECT id, name, user_data_id, user_data_link_policy FROM cloud.vm_template WHERE user_data_id = 1 AND removed IS NULL; - If no templates, update user_data_id to NULL, in vm_template table - use update sql below.
UPDATE cloud.vm_template SET user_data_id = NULL, user_data_link_policy = NULL WHERE user_data_id = 1 AND removed IS NOT NULL; - Now, Delete the user data from UI, or use delete sql below
DELETE FROM cloud.user_data WHERE id = 1;
Hope this helps.
