uyuni icon indicating copy to clipboard operation
uyuni copied to clipboard

Package refresh not working on Xen-host minions

Open ftigges-comdev opened this issue 1 year ago • 6 comments

Problem description

When asking for a refresh, there is the following error in the event list: Client-Ausführung meldet "An unexpected error has occurred. Please check the server logs." (code -1)"

Steps to reproduce

  1. Login
  2. Go to system
  3. Tab: Software
  4. Refresh package list. (Button right corner)

Uyuni version

2024.01-230900.212.1.uyuni3

Uyuni proxy version (if used)

---

Useful logs

2024-02-14 13:15:10,551 [salt-event-thread-4] ERROR com.suse.manager.webui.services.SaltServerActionService - Error processing Salt job return
org.hibernate.PropertyAccessException: Exception occurred inside setter of com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.detailsAsString
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:68) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.tuple.entity.AbstractEntityTuplizer.setPropertyValues(AbstractEntityTuplizer.java:661) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.tuple.entity.PojoEntityTuplizer.setPropertyValues(PojoEntityTuplizer.java:146) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.persister.entity.AbstractEntityPersister.setPropertyValues(AbstractEntityPersister.java:4985) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:217) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:160) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.process.internal.AbstractRowReader.performTwoPhaseLoad(AbstractRowReader.java:252) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.process.internal.AbstractRowReader.finishUp(AbstractRowReader.java:215) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.process.internal.ResultSetProcessorImpl.extractResults(ResultSetProcessorImpl.java:133) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:107) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.entity.plan.AbstractLoadPlanBasedEntityLoader.load(AbstractLoadPlanBasedEntityLoader.java:191) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:4290) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:604) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:572) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:226) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.doOnLoad(DefaultLoadEventListener.java:122) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:93) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.internal.SessionImpl.fireLoadNoChecks(SessionImpl.java:1277) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.internal.SessionImpl.immediateLoad(SessionImpl.java:1119) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:184) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:315) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.pojo.bytebuddy.ByteBuddyInterceptor.intercept(ByteBuddyInterceptor.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.ProxyConfiguration$InterceptorDispatcher.intercept(ProxyConfiguration.java:95) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at com.redhat.rhn.domain.action.Action$HibernateProxy$eLkU6W04.getActionType(Unknown Source) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.shouldCleanupAction(SaltUtils.java:2030) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.handleUptimeUpdate(SaltUtils.java:2013) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.handlePackageProfileUpdate(SaltUtils.java:1483) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.lambda$updateServerAction$25(SaltUtils.java:623) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.utils.SaltUtils.updateServerAction(SaltUtils.java:623) ~[rhn.jar:?]
	at com.suse.manager.webui.services.SaltServerActionService.lambda$handleAction$146(SaltServerActionService.java:2674) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.webui.services.SaltServerActionService.lambda$handleAction$147(SaltServerActionService.java:2652) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.webui.services.SaltServerActionService.handleAction(SaltServerActionService.java:2645) ~[rhn.jar:?]
	at com.suse.manager.reactor.messaging.JobReturnEventMessageAction.lambda$execute$4(JobReturnEventMessageAction.java:137) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.reactor.messaging.JobReturnEventMessageAction.lambda$execute$5(JobReturnEventMessageAction.java:136) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.reactor.messaging.JobReturnEventMessageAction.execute(JobReturnEventMessageAction.java:136) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventListener.lambda$notify$1(PGEventListener.java:76) ~[rhn.jar:?]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
	at com.suse.manager.reactor.PGEventListener.lambda$notify$2(PGEventListener.java:74) ~[rhn.jar:?]
	at java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) ~[?:?]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
	at com.suse.manager.reactor.PGEventListener.notify(PGEventListener.java:71) ~[rhn.jar:?]
	at com.suse.salt.netapi.event.AbstractEventStream.notifyListeners(AbstractEventStream.java:64) ~[salt-netapi-clientsalt-netapi-client.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$processEvents$6(PGEventStream.java:222) ~[rhn.jar:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
	at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[?:?]
	at java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
	at com.suse.manager.reactor.PGEventStream.processEvents(PGEventStream.java:216) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$notification$1(PGEventStream.java:198) ~[rhn.jar:?]
	at com.redhat.rhn.frontend.events.TransactionHelper.run(TransactionHelper.java:63) ~[rhn.jar:?]
	at com.redhat.rhn.frontend.events.TransactionHelper.handlingTransaction(TransactionHelper.java:48) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$notification$3(PGEventStream.java:197) ~[rhn.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.reflect.InvocationTargetException
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	... 65 more
Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected STRING but was BEGIN_OBJECT at path $
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:226) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:954) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:919) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:868) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.virtualization.GuestCreateDetails.parse(GuestCreateDetails.java:55) ~[rhn.jar:?]
	at com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.setDetailsAsString(VirtualizationCreateGuestAction.java:73) ~[rhn.jar:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	... 65 more
Caused by: java.lang.IllegalStateException: Expected STRING but was BEGIN_OBJECT at path $
	at com.google.gson.internal.bind.JsonTreeReader.nextString(JsonTreeReader.java:181) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.reactor.utils.LocalDateTimeISOAdapter.read(LocalDateTimeISOAdapter.java:51) ~[rhn.jar:?]
	at com.suse.manager.reactor.utils.LocalDateTimeISOAdapter.read(LocalDateTimeISOAdapter.java:34) ~[rhn.jar:?]
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.reactor.utils.OptionalTypeAdapterFactory$1.read(OptionalTypeAdapterFactory.java:65) ~[rhn.jar:?]
	at com.suse.manager.reactor.utils.OptionalTypeAdapterFactory$1.read(OptionalTypeAdapterFactory.java:55) ~[rhn.jar:?]
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:954) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:919) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:868) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.virtualization.GuestCreateDetails.parse(GuestCreateDetails.java:55) ~[rhn.jar:?]
	at com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.setDetailsAsString(VirtualizationCreateGuestAction.java:73) ~[rhn.jar:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	... 65 more
2024-02-14 13:15:10,569 [salt-event-thread-4] ERROR com.suse.manager.reactor.PGEventListener - Unexpected exception while executing a MessageAction
org.hibernate.PropertyAccessException: Exception occurred inside setter of com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.detailsAsString
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:68) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.tuple.entity.AbstractEntityTuplizer.setPropertyValues(AbstractEntityTuplizer.java:661) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.tuple.entity.PojoEntityTuplizer.setPropertyValues(PojoEntityTuplizer.java:146) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.persister.entity.AbstractEntityPersister.setPropertyValues(AbstractEntityPersister.java:4985) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.engine.internal.TwoPhaseLoad.doInitializeEntity(TwoPhaseLoad.java:217) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.engine.internal.TwoPhaseLoad.initializeEntity(TwoPhaseLoad.java:160) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.process.internal.AbstractRowReader.performTwoPhaseLoad(AbstractRowReader.java:252) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.process.internal.AbstractRowReader.finishUp(AbstractRowReader.java:215) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.process.internal.ResultSetProcessorImpl.extractResults(ResultSetProcessorImpl.java:133) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.plan.exec.internal.AbstractLoadPlanBasedLoader.executeLoad(AbstractLoadPlanBasedLoader.java:107) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.loader.entity.plan.AbstractLoadPlanBasedEntityLoader.load(AbstractLoadPlanBasedEntityLoader.java:191) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:4290) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:604) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:572) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:226) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.doOnLoad(DefaultLoadEventListener.java:122) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:93) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.internal.SessionImpl.fireLoadNoChecks(SessionImpl.java:1277) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.internal.SessionImpl.immediateLoad(SessionImpl.java:1119) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:184) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:315) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.pojo.bytebuddy.ByteBuddyInterceptor.intercept(ByteBuddyInterceptor.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at org.hibernate.proxy.ProxyConfiguration$InterceptorDispatcher.intercept(ProxyConfiguration.java:95) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	at com.redhat.rhn.domain.action.Action$HibernateProxy$eLkU6W04.getActionType(Unknown Source) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.shouldCleanupAction(SaltUtils.java:2030) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.handleUptimeUpdate(SaltUtils.java:2013) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.handlePackageProfileUpdate(SaltUtils.java:1483) ~[rhn.jar:?]
	at com.suse.manager.utils.SaltUtils.lambda$updateServerAction$25(SaltUtils.java:623) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.utils.SaltUtils.updateServerAction(SaltUtils.java:623) ~[rhn.jar:?]
	at com.suse.manager.webui.services.SaltServerActionService.lambda$handleAction$146(SaltServerActionService.java:2674) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.webui.services.SaltServerActionService.lambda$handleAction$147(SaltServerActionService.java:2652) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.webui.services.SaltServerActionService.handleAction(SaltServerActionService.java:2645) ~[rhn.jar:?]
	at com.suse.manager.reactor.messaging.JobReturnEventMessageAction.lambda$execute$4(JobReturnEventMessageAction.java:137) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.reactor.messaging.JobReturnEventMessageAction.lambda$execute$5(JobReturnEventMessageAction.java:136) ~[rhn.jar:?]
	at java.util.Optional.ifPresent(Optional.java:183) ~[?:?]
	at com.suse.manager.reactor.messaging.JobReturnEventMessageAction.execute(JobReturnEventMessageAction.java:136) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventListener.lambda$notify$1(PGEventListener.java:76) ~[rhn.jar:?]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
	at com.suse.manager.reactor.PGEventListener.lambda$notify$2(PGEventListener.java:74) ~[rhn.jar:?]
	at java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) ~[?:?]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
	at com.suse.manager.reactor.PGEventListener.notify(PGEventListener.java:71) ~[rhn.jar:?]
	at com.suse.salt.netapi.event.AbstractEventStream.notifyListeners(AbstractEventStream.java:64) ~[salt-netapi-clientsalt-netapi-client.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$processEvents$6(PGEventStream.java:222) ~[rhn.jar:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
	at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[?:?]
	at java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
	at com.suse.manager.reactor.PGEventStream.processEvents(PGEventStream.java:216) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$notification$1(PGEventStream.java:198) ~[rhn.jar:?]
	at com.redhat.rhn.frontend.events.TransactionHelper.run(TransactionHelper.java:63) ~[rhn.jar:?]
	at com.redhat.rhn.frontend.events.TransactionHelper.handlingTransaction(TransactionHelper.java:48) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$notification$3(PGEventStream.java:197) ~[rhn.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
Caused by: java.lang.reflect.InvocationTargetException
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	... 65 more
Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected STRING but was BEGIN_OBJECT at path $
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:226) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:954) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:919) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:868) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.virtualization.GuestCreateDetails.parse(GuestCreateDetails.java:55) ~[rhn.jar:?]
	at com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.setDetailsAsString(VirtualizationCreateGuestAction.java:73) ~[rhn.jar:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	... 65 more
Caused by: java.lang.IllegalStateException: Expected STRING but was BEGIN_OBJECT at path $
	at com.google.gson.internal.bind.JsonTreeReader.nextString(JsonTreeReader.java:181) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.reactor.utils.LocalDateTimeISOAdapter.read(LocalDateTimeISOAdapter.java:51) ~[rhn.jar:?]
	at com.suse.manager.reactor.utils.LocalDateTimeISOAdapter.read(LocalDateTimeISOAdapter.java:34) ~[rhn.jar:?]
	at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:285) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.reactor.utils.OptionalTypeAdapterFactory$1.read(OptionalTypeAdapterFactory.java:65) ~[rhn.jar:?]
	at com.suse.manager.reactor.utils.OptionalTypeAdapterFactory$1.read(OptionalTypeAdapterFactory.java:55) ~[rhn.jar:?]
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:954) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:919) ~[google-gson_google-gsongson.jar:?]
	at com.google.gson.Gson.fromJson(Gson.java:868) ~[google-gson_google-gsongson.jar:?]
	at com.suse.manager.virtualization.GuestCreateDetails.parse(GuestCreateDetails.java:55) ~[rhn.jar:?]
	at com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.setDetailsAsString(VirtualizationCreateGuestAction.java:73) ~[rhn.jar:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.hibernate.property.access.spi.SetterMethodImpl.set(SetterMethodImpl.java:45) ~[hibernate5_hibernate-core.jar:5.3.25.Final]
	... 65 more
2024-02-14 13:15:10,570 [salt-event-thread-4] ERROR com.redhat.rhn.frontend.events.TransactionHelper - null
com.suse.manager.reactor.PGEventListenerException: null
	at com.suse.manager.reactor.PGEventListener.lambda$notify$1(PGEventListener.java:80) ~[rhn.jar:?]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
	at com.suse.manager.reactor.PGEventListener.lambda$notify$2(PGEventListener.java:74) ~[rhn.jar:?]
	at java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) ~[?:?]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?]
	at com.suse.manager.reactor.PGEventListener.notify(PGEventListener.java:71) ~[rhn.jar:?]
	at com.suse.salt.netapi.event.AbstractEventStream.notifyListeners(AbstractEventStream.java:64) ~[salt-netapi-clientsalt-netapi-client.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$processEvents$6(PGEventStream.java:222) ~[rhn.jar:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?]
	at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[?:?]
	at java.util.stream.Sink$ChainedReference.end(Sink.java:258) ~[?:?]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[?:?]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?]
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?]
	at com.suse.manager.reactor.PGEventStream.processEvents(PGEventStream.java:216) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$notification$1(PGEventStream.java:198) ~[rhn.jar:?]
	at com.redhat.rhn.frontend.events.TransactionHelper.run(TransactionHelper.java:63) ~[rhn.jar:?]
	at com.redhat.rhn.frontend.events.TransactionHelper.handlingTransaction(TransactionHelper.java:48) ~[rhn.jar:?]
	at com.suse.manager.reactor.PGEventStream.lambda$notification$3(PGEventStream.java:197) ~[rhn.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
2024-02-14 13:15:10,571 [salt-event-thread-4] ERROR com.suse.manager.reactor.PGEventStream - Events [180990] were lost

Additional information

Only Xen hosts are affected. The virtualization formula is active on all five systems. There are no obvious differences. Virtualization tab is showing clients and states.

Updating packages and applying highstates works as expected.

ftigges-comdev avatar Feb 14 '24 12:02 ftigges-comdev

Hi there!

Poking in the dark here but could you please try spacecmd -d system_schedulepackagerefresh <Xen system name as shown in Uyuni Web UI> on your Uyuni server and post the debug messages?

system_schedulepackagerefresh is the command line equivalent to Package List Refresh. The option -d enables the debug mode and maybe this gives us more clues : )

cFabij avatar Feb 20 '24 07:02 cFabij

Hi, thanks for your time.

Package refresh is starting but same error as before.

spacecmd -d -s uyuni.domain.tld system_schedulepackagerefresh xen-host-1 DEBUG: command=, return_value=False DEBUG: Read configuration from /root/.spacecmd/config DEBUG: Loading configuration section [spacecmd] DEBUG: Current Configuration: {'server': 'uyuni.domain.tld'} DEBUG: Configuration section [uyuni.domain.tld] does not exist DEBUG: Connecting to https://uyuni.domain.tld/rpc/api DEBUG: Server API Version = 26 DEBUG: Using cached credentials from /root/.spacecmd/uyuni.domain.tld/session DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/ssm DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/ssm does not exist DEBUG: command=, return_value=False DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/systems DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/errata DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/errata does not exist DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/packages_short DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/packages_short does not exist DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/packages_long DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/packages_long does not exist DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/packages_by_id DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/packages_by_id does not exist INFO: Connected to https://uyuni.domain.tld/rpc/api as admin DEBUG: command=system_schedulepackagerefresh xen-host-1, return_value=0

Different Xen-Host: Package refresh is starting and completing. No error.

spacecmd -d -s uyuni.domain.tld system_schedulepackagerefresh xen-host-2 DEBUG: command=, return_value=False DEBUG: Read configuration from /root/.spacecmd/config DEBUG: Loading configuration section [spacecmd] DEBUG: Current Configuration: {'server': 'uyuni.domain.tld'} DEBUG: Configuration section [uyuni.domain.tld] does not exist DEBUG: Connecting to https://uyuni.domain.tld/rpc/api DEBUG: Server API Version = 26 DEBUG: Using cached credentials from /root/.spacecmd/uyuni.domain.tld/session DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/ssm DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/ssm does not exist DEBUG: command=, return_value=False DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/systems DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/errata DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/errata does not exist DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/packages_short DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/packages_short does not exist DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/packages_long DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/packages_long does not exist DEBUG: Loading cache from /root/.spacecmd/uyuni.domain.tld/admin/packages_by_id DEBUG: /root/.spacecmd/uyuni.domain.tld/admin/packages_by_id does not exist INFO: Connected to https://uyuni.domain.tld/rpc/api as admin DEBUG: command=system_schedulepackagerefresh xen-host-2, return_value=0

ftigges-comdev avatar Feb 20 '24 08:02 ftigges-comdev

Things I tried today

  • Removed salt key - reactivated system
  • Removed temp files on minion

In regard of error: MessageAction org.hibernate.PropertyAccessException: Exception occurred inside setter of com.redhat.rhn.domain.action.virtualization.VirtualizationCreateGuestAction.detailsAsString

  • Removed minion from group with virtualization formula.
  • Removed system entitlement virtualization

Same error msg.

ftigges-comdev avatar Feb 20 '24 10:02 ftigges-comdev

I assume it's a minion problem because the above system_schedulepackagerefresh exited with a 0 (success) return code.

Could you check/post your repository file? (E.g. /etc/apt/sources.list and /etc/apt/sources.list.d/susemanager:channels.list on Ubuntu)

All regular repositories should be commented out and only Uyuni repositories should be active.

Maybe there is also some info in /var/log/venv-salt-minion.log.

cFabij avatar Feb 20 '24 11:02 cFabij

Only SuseManager channel is enabled. $ zypper lr | grep Yes 16 | susemanager:opensuse_leap15_5-x86_64 | opensuse_leap15_5-x86_64 | Yes | ( ) No | Yes

There are no new entries in /var/log/venv-salt-minion.log when refresh is requested.

zypper.log main.cc(~Bye):87 ===== Exiting main(0) =====

On a side note: The uptime isn't reported correctly. 12 day on web UI vs 4 days on local system.

ftigges-comdev avatar Feb 20 '24 14:02 ftigges-comdev

Deleted the profile, bootstrapped the system and everything works as expected.

There are still two systems with this problem.

Close at own discretion.

Thanks again for your time.

ftigges-comdev avatar Feb 20 '24 14:02 ftigges-comdev

Seems to be solved for now, please feel free to reopen if the problem reappears :smiley:

avshiliaev avatar May 21 '24 15:05 avshiliaev