Horreum icon indicating copy to clipboard operation
Horreum copied to clipboard

Published 20 hours ago • verified publisher icon Hyperfoil

Feature: Datastore integration with `Collector API`

Open johnaohara opened this issue 1 year ago • 10 comments

An external Collector API tool (https://github.com/Karm/collector) allows performance metrics to be pushed and queried as JSON documents via a REST API

In order to integrate existing datasets into Horreum for analysis, a new Collector API datastore needs to be created that allows Horreum Users to query and existing instance.

The Collector API datastore should be defined as a new Datastore Type (https://github.com/Hyperfoil/Horreum/blob/master/horreum-backend/src/main/java/io/hyperfoil/tools/horreum/datastore/Datastore.java), similar to how the Elasticsearch datastore is integrated (https://github.com/Hyperfoil/Horreum/blob/master/horreum-backend/src/main/java/io/hyperfoil/tools/horreum/datastore/ElasticsearchDatastore.java)

Documentation on how the Elasticsearch document store is used can be found here: https://horreum.hyperfoil.io/docs/tutorials/query-elasticserach/

johnaohara avatar Jan 18 '24 10:01 johnaohara

Thx.

Karm avatar Jan 18 '24 10:01 Karm

Hello @johnaohara , I have drafted a rough CollectorAPI datastore that gets a json array (as an ArrayNode) from the Collector API and creates a new DatastoreResponse but I am getting:

2024-04-17 17:23:24,926 ERROR [io.hyp.too.hor.svc.RunServiceImpl] (executor-thread-4) Failed to persist run.: jakarta.persistence.TransactionRequiredException: Transaction is not active, consider adding @Transactional to your method to automatically activate one.
	at io.quarkus.hibernate.orm.runtime.session.TransactionScopedSession.persist(TransactionScopedSession.java:143)
	at org.hibernate.engine.spi.SessionLazyDelegator.persist(SessionLazyDelegator.java:275)
	at org.hibernate.Session_OpdLahisOZ9nWRPXMsEFQmQU03A_Synthetic_ClientProxy.persist(Unknown Source)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl.addAuthenticated(RunServiceImpl.java:675)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl.getPersistRun(RunServiceImpl.java:628)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl.lambda$addRunFromData$4(RunServiceImpl.java:575)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl.addRunFromData(RunServiceImpl.java:574)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass.addRunFromData$$superforward(Unknown Source)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass$$function$$199.apply(Unknown Source)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
	at io.hyperfoil.tools.horreum.server.TokenInterceptor.wrap(TokenInterceptor.java:49)
	at io.hyperfoil.tools.horreum.server.TokenInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
	at io.hyperfoil.tools.horreum.server.RolesInterceptor.intercept(RolesInterceptor.java:70)
	at io.hyperfoil.tools.horreum.server.RolesInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
	at io.quarkus.security.runtime.interceptor.SecurityHandler.handle(SecurityHandler.java:47)
	at io.quarkus.security.runtime.interceptor.PermitAllInterceptor.intercept(PermitAllInterceptor.java:23)
	at io.quarkus.security.runtime.interceptor.PermitAllInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:136)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
	at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:44)
	at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_PermitAllInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass.addRunFromData(Unknown Source)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl.addRunFromData(RunServiceImpl.java:494)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl_ClientProxy.addRunFromData(Unknown Source)
	at io.hyperfoil.tools.horreum.api.services.RunService$quarkusrestinvoker$addRunFromData_ab145475c68a03706378ea6d090426ed8639d50c.invoke(Unknown Source)
	at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
	at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
	at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:1583)

However, addRunFromData is already annotated with @Transactional and I don't see how the new DataStore could affect this anyway. Any help would be welcome.

Furthermore, looking at what the new DataStore is doing, it's essentially getting the data from the Collector and putting them in Horreum, i.e. it duplicates the data while I thought it would use them on demand. @Karm is that what you had in mind? In that case what's the benefit of keeping the collector around instead of hosting a Horreum instance instead and uploading directly to it?

zakkak avatar Apr 17 '24 14:04 zakkak

Hey @zakkak that error is not expected, and this code path is well tested :( Are there any other error messages before that, i.e. did anything cause a transaction rollback / abort ?

johnaohara avatar Apr 18 '24 12:04 johnaohara

Furthermore, looking at what the new DataStore is doing, it's essentially getting the data from the Collector and putting them in Horreum, i.e. it duplicates the data while I thought it would use them on demand. @Karm is that what you had in mind? In that case what's the benefit of keeping the collector around instead of hosting a Horreum instance instead and uploading directly to it?

Horreum allows users to dynamically create Labels (i.e. derived data/metrics) retrospectively after data has been "indexed". e.g. if a user adds a new change detection variable, or a the calculation for a Labels changes, Horreum can retrospectively calculate values from all retrieved documents.

The calculation of Labels occurs within the database, and therefore we are reliant on data being in the backend store before it is processed.

If order to handle Label calculation for downstream datastores, we cache the original document. This allows us to process the JSON documents, but also saves us from having to retrieve all historic documents if the user defines a new or updates a Label

johnaohara avatar Apr 18 '24 12:04 johnaohara

Hey @zakkak that error is not expected, and this code path is well tested :( Are there any other error messages before that, i.e. did anything cause a transaction rollback / abort ?

Not that I saw, I will try to reproduce and have a better look.

zakkak avatar Apr 18 '24 12:04 zakkak

Hi @zakkak I am coming back to this, the error is strange and I have not seen this happen before. All the code paths that lead to that method call io.hyperfoil.tools.horreum.svc.RunServiceImpl.addAuthenticated(RunServiceImpl.java:675) are wrapped in a transaction. Have you managed to re-create this problem? are you still seeing it?

johnaohara avatar May 09 '24 16:05 johnaohara

Hi @johnaohara, sorry for the late reply I was on vacation.

No, I didn't have the time to reproduce yet. I am aware that you are going to give it a try yourself, let me know if you need anuthing more.

zakkak avatar May 14 '24 12:05 zakkak

Hi @johnaohara, I am trying this out again and am now getting a different error when trying to import some runs (this time on a Linux machine):

Logs: 
2024-06-12 10:48:36,683 WARN  [com.arj.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check processing TX 0:ffffc0a802b2:b22d:66694b32:2972 in state  RUN
2024-06-12 10:48:36,684 WARN  [com.arj.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffffc0a802b2:b22d:66694b32:2972 invoked while multiple threads active within it.
2024-06-12 10:48:36,685 WARN  [com.arj.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012381: Action id 0:ffffc0a802b2:b22d:66694b32:2972 completed with multiple threads - thread executor-thread-23 was in progress with org.hibernate.engine.internal.ManagedTypeHelper.asPersistentAttributeInterceptable(ManagedTypeHelper.java:267)
org.hibernate.bytecode.internal.BytecodeEnhancementMetadataPojoImpl.extractLazyInterceptor(BytecodeEnhancementMetadataPojoImpl.java:277)
org.hibernate.bytecode.internal.BytecodeEnhancementMetadataPojoImpl.hasUnFetchedAttributes(BytecodeEnhancementMetadataPojoImpl.java:113)
org.hibernate.persister.entity.AbstractEntityPersister.hasUninitializedLazyProperties(AbstractEntityPersister.java:4384)
org.hibernate.engine.internal.Cascade.cascade(Cascade.java:98)
org.hibernate.event.internal.AbstractSaveEventListener.cascadeAfterSave(AbstractSaveEventListener.java:490)
org.hibernate.event.internal.DefaultPersistEventListener.justCascade(DefaultPersistEventListener.java:161)
org.hibernate.event.internal.DefaultPersistEventListener.entityIsPersistent(DefaultPersistEventListener.java:154)
org.hibernate.event.internal.DefaultPersistEventListener.persist(DefaultPersistEventListener.java:90)
org.hibernate.event.internal.DefaultPersistEventListener.onPersist(DefaultPersistEventListener.java:77)
org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:138)
org.hibernate.internal.SessionImpl.persistOnFlush(SessionImpl.java:819)
org.hibernate.engine.spi.CascadingActions$8.cascade(CascadingActions.java:333)
org.hibernate.engine.spi.CascadingActions$8.cascade(CascadingActions.java:323)
org.hibernate.engine.internal.Cascade.cascadeToOne(Cascade.java:518)
org.hibernate.engine.internal.Cascade.cascadeAssociation(Cascade.java:439)
org.hibernate.engine.internal.Cascade.cascadeProperty(Cascade.java:225)
org.hibernate.engine.internal.Cascade.cascade(Cascade.java:158)
org.hibernate.event.internal.AbstractFlushingEventListener.cascadeOnFlush(AbstractFlushingEventListener.java:161)
org.hibernate.event.internal.AbstractFlushingEventListener.prepareEntityFlushes(AbstractFlushingEventListener.java:144)
org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:79)
org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:38)
org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1412)
org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1398)
org.hibernate.query.sql.internal.NativeQueryImpl.prepareForExecution(NativeQueryImpl.java:599)
org.hibernate.query.spi.AbstractSelectionQuery.beforeQuery(AbstractSelectionQuery.java:391)
org.hibernate.query.spi.AbstractSelectionQuery.list(AbstractSelectionQuery.java:364)
org.hibernate.query.spi.AbstractSelectionQuery.getSingleResult(AbstractSelectionQuery.java:473)
io.hyperfoil.tools.horreum.server.RoleManager.setRoles(RoleManager.java:28)
io.hyperfoil.tools.horreum.server.RoleManager_ClientProxy.setRoles(Unknown Source)
io.hyperfoil.tools.horreum.server.RolesInterceptor.intercept(RolesInterceptor.java:78)
io.hyperfoil.tools.horreum.server.RolesInterceptor_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInCallerTx(TransactionalInterceptorBase.java:335)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:40)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass.transform(Unknown Source)
io.hyperfoil.tools.horreum.svc.RunServiceImpl.addAuthenticated(RunServiceImpl.java:688)
io.hyperfoil.tools.horreum.svc.RunServiceImpl.getPersistRun(RunServiceImpl.java:628)
io.hyperfoil.tools.horreum.svc.RunServiceImpl.lambda$addRunFromData$4(RunServiceImpl.java:575)
java.base/java.lang.Iterable.forEach(Iterable.java:75)
io.hyperfoil.tools.horreum.svc.RunServiceImpl.addRunFromData(RunServiceImpl.java:574)
io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass.addRunFromData$$superforward(Unknown Source)
io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass$$function$$10.apply(Unknown Source)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
io.hyperfoil.tools.horreum.server.TokenInterceptor.wrap(TokenInterceptor.java:49)
io.hyperfoil.tools.horreum.server.TokenInterceptor_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
io.hyperfoil.tools.horreum.server.RolesInterceptor.intercept(RolesInterceptor.java:70)
io.hyperfoil.tools.horreum.server.RolesInterceptor_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
io.quarkus.security.runtime.interceptor.SecurityHandler.handle(SecurityHandler.java:47)
io.quarkus.security.runtime.interceptor.PermitAllInterceptor.intercept(PermitAllInterceptor.java:23)
io.quarkus.security.runtime.interceptor.PermitAllInterceptor_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:136)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:44)
io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_PermitAllInterceptor_Bean.intercept(Unknown Source)
io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass.addRunFromData(Unknown Source)
io.hyperfoil.tools.horreum.svc.RunServiceImpl.addRunFromData(RunServiceImpl.java:494)
io.hyperfoil.tools.horreum.svc.RunServiceImpl_ClientProxy.addRunFromData(Unknown Source)
io.hyperfoil.tools.horreum.api.services.RunService$quarkusrestinvoker$addRunFromData_ab145475c68a03706378ea6d090426ed8639d50c.invoke(Unknown Source)
org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
java.base/java.lang.Thread.run(Thread.java:1583)


2024-06-12 10:48:36,686 WARN  [com.arj.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012108: CheckedAction::check - atomic action 0:ffffc0a802b2:b22d:66694b32:2972 aborting with 1 threads active!
2024-06-12 10:48:36,686 WARN  [org.hib.res.tra.bac.jta.int.syn.SynchronizationCallbackCoordinatorTrackingImpl] (Transaction Reaper Worker 0) HHH000451: Transaction afterCompletion called by a background thread; delaying afterCompletion processing until the original thread can handle it. [status=4]
2024-06-12 10:48:36,686 WARN  [com.arj.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[#456,Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffffc0a802b2:b22d:66694b32:2972
2024-06-12 10:48:36,690 WARN  [com.arj.ats.arjuna] (executor-thread-23) ARJUNA012077: Abort called on already aborted atomic action 0:ffffc0a802b2:b22d:66694b32:2972
2024-06-12 10:48:36,691 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-23) HTTP Request to /api/run/data?start=2023-11-15T10%3A11%3A43%2B00%3A00&stop=2024-06-12T10%3A11%3A43%2B00%3A00&test=Quarkus%20main&owner=dev-team&access=PUBLIC failed, error id: 1b6edb27-08f3-4e4d-ab71-4b630bb76d3b-8: io.quarkus.arc.ArcUndeclaredThrowableException: Error invoking subclass method
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl_Subclass.addRunFromData(Unknown Source)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl.addRunFromData(RunServiceImpl.java:494)
	at io.hyperfoil.tools.horreum.svc.RunServiceImpl_ClientProxy.addRunFromData(Unknown Source)
	at io.hyperfoil.tools.horreum.api.services.RunService$quarkusrestinvoker$addRunFromData_ab145475c68a03706378ea6d090426ed8639d50c.invoke(Unknown Source)
	at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
	at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:141)
	at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:147)
	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: jakarta.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffffc0a802b2:b22d:66694b32:2972
	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1261)
	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:104)
	at io.quarkus.narayana.jta.runtime.NotifyingTransactionManager.commit(NotifyingTransactionManager.java:70)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.endTransaction(TransactionalInterceptorBase.java:406)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:175)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
	at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
	at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor.intercept(StandardSecurityCheckInterceptor.java:44)
	at io.quarkus.resteasy.reactive.server.runtime.StandardSecurityCheckInterceptor_PermitAllInterceptor_Bean.intercept(Unknown Source)
	at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
	at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
	at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
	... 14 more

What I did:

  1. Started Horreum in dev mode (without testing) from my branch in https://github.com/Hyperfoil/Horreum/pull/1703
  2. Setup the Collector integration (added the URL and TOKEN, no username or password)
  3. Created a JWT token
  4. Run
curl "http://localhost:8080/api/run/data?start=2023-11-15T10%3A11%3A43%2B00%3A00&stop=2024-06-12T10%3A11%3A43%2B00%3A00&test=Quarkus%20main&owner=dev-team&access=PUBLIC" -s -H 'content-type: application/json' -H "Authorization: Bearer ${BEARER_TOKEN}" -d '{ "tag": "quarkus-main-ci" }'

to import the runs of the quarkus-main-ci tag (which contains an array of 7126 elements/runs).

Any hints on what I might be doing wrong?

zakkak avatar Jun 12 '24 08:06 zakkak

HI @zakkak you are not doing anything wrong, it is just that all the documents are being processed by a single blocking call within a single transaction, and that transaction is timing out.

For now, you can increase the tx timeout : https://quarkus.io/guides/transaction#configuring-the-transaction-timeout

This is a short term workaround, and will not scale for large number of runs.

I had started working on change that offloads the processing to a message queue, I will open a PR for that so we can get the changes in at the same time

johnaohara avatar Jun 12 '24 09:06 johnaohara

Thank you @johnaohara I will do that.

Note that it shouldn't be necessary after the initial load of the data as I expect us to push the new builds on a daily basis after that, in which case the timeout could be enough.

zakkak avatar Jun 12 '24 09:06 zakkak