drill4j icon indicating copy to clipboard operation
drill4j copied to clipboard

Published 20 hours ago • verified publisher icon Drill4J

Fatal exception in coroutines machinery for DispatchedContinuation with DRILL_PLUGINS_REMOTE_ENABLED=false

Open Aliaksei-Tatarynchyk opened this issue 2 years ago • 0 comments

Hello,

I started drill4j admin back-end and front-end, added -agentpath into my Spring WebFlux app run configuration and got this error during startup of application:

2022-05-16 22:42:44:231 [MACOSX][INFO][MainLogger] The native agent was loaded 
2022-05-16 22:42:44:231 [MACOSX][INFO][MainLogger] Pid is: 37125 
2022-05-16 22:42:45:546 [MACOSX][WARN][jvm api] GetSystemProperty: The functionality is not available in this virtual machine. 
2022-05-16 22:42:45:546 [MACOSX][INFO][VmInitEvent] run with http hook 
Configuration for unix
2022-05-16 22:42:45:687 [MACOSX][INFO][VmInitEvent] Agent is not alive. Waiting for package settings from ws://localhost:8090... 
2022-05-16 22:42:45:962 [MACOSX][INFO][DrillWebsocket] Agent connected 
2022-05-16 22:42:46:195 [MACOSX][INFO][VmInitEvent] Agent is not alive. Waiting for package settings from ws://localhost:8090... 
2022-05-16 22:42:46:705 [MACOSX][INFO][VmInitEvent] Agent is not alive. Waiting for package settings from ws://localhost:8090... 
2022-05-16 22:42:47:215 [MACOSX][INFO][VmInitEvent] Agent is not alive. Waiting for package settings from ws://localhost:8090... 
2022-05-16 22:42:47:717 [MACOSX][INFO][VmInitEvent] Agent is not alive. Waiting for package settings from ws://localhost:8090... 
2022-05-16 22:42:47:854 [MACOSX][INFO][tempTopicLogger] Agent got a new headerMapping:  
Uncaught Kotlin exception: kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[WorkerCoroutineDispatcherImpl@323fbf78, Continuation @ $run$lambda-0COROUTINE$16]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
2022-05-16 22:42:48:219 [MACOSX][INFO][VmInitEvent] No plugins loaded from ws://localhost:8090. 
    at 0   libdrill_agent.dylib                0x000000010a1a44ee kfun:kotlin.Error#<init>(kotlin.String?;kotlin.Throwable?){} + 126 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/Exceptions.kt:14:63)
    at 1   libdrill_agent.dylib                0x000000010a34bdee kfun:kotlinx.coroutines.CoroutinesInternalError#<init>(kotlin.String;kotlin.Throwable){} + 126 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Exceptions.common.kt:28:77)
    at 2   libdrill_agent.dylib                0x000000010a370e87 kfun:kotlinx.coroutines.DispatchedTask#handleFatalException(kotlin.Throwable?;kotlin.Throwable?){} + 887 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:144:22)
    at 3   libdrill_agent.dylib                0x000000010a370aad kfun:kotlinx.coroutines.DispatchedTask#run(){} + 3533 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:115:13)
    at 4   libdrill_agent.dylib                0x000000010a34711f kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long + 831 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274:18)
    at 5   libdrill_agent.dylib                0x000000010a385382 kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} + 898 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:80:40)
    at 6   libdrill_agent.dylib                0x000000010a38e30c kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 412 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Workers.kt:49:17)
    at 7   libdrill_agent.dylib                0x000000010a38e500 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$171.invoke#internal + 64 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Workers.kt:47:24)
    at 8   libdrill_agent.dylib                0x000000010a38e560 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$171.$<bridge-UNN>invoke(){}#internal + 64 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Workers.kt:47:24)
    at 9   libdrill_agent.dylib                0x000000010a1db8a9 WorkerLaunchpad + 185 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:69:54)
    at 10  libdrill_agent.dylib                0x000000010a5fc61f _ZN6Worker19processQueueElementEb + 3135
    at 11  libdrill_agent.dylib                0x000000010a5fb9c6 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
    at 12  libsystem_pthread.dylib             0x00007fff2060a8fc _pthread_start + 224
    at 13  libsystem_pthread.dylib             0x00007fff20606443 thread_start + 15
Caused by: kotlin.IllegalArgumentException: Coroutines must be initialized from the main thread: call 'initMainThread' from the main thread first
    at 0   libdrill_agent.dylib                0x000000010a1aad7f kfun:kotlin.Throwable#<init>(kotlin.String?){} + 95 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
    at 1   libdrill_agent.dylib                0x000000010a1a39dd kfun:kotlin.Exception#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
    at 2   libdrill_agent.dylib                0x000000010a1a3bbd kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
    at 3   libdrill_agent.dylib                0x000000010a1a3ecd kfun:kotlin.IllegalArgumentException#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/Exceptions.kt:59:44)
    at 4   libdrill_agent.dylib                0x000000010a396ec1 kfun:kotlinx.coroutines.getOrCreateMainThread#internal + 705 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Thread.kt:36:6)
    at 5   libdrill_agent.dylib                0x000000010a396b62 kfun:kotlinx.coroutines#<get-mainThread>(){}kotlinx.coroutines.MainThread + 226 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Thread.kt:31:66)
    at 6   libdrill_agent.dylib                0x000000010a39604a kfun:kotlinx.coroutines#[email protected](){}kotlinx.coroutines.Thread + 170 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/nativeDarwin/src/Thread.kt:26:17)
    at 7   libdrill_agent.dylib                0x000000010a390127 kfun:kotlinx.coroutines.internal.ShareableObject#<get-ownerThreadOrNull>(){}kotlinx.coroutines.Thread? + 583 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/Sharing.kt:137:37)
    at 8   libdrill_agent.dylib                0x000000010a38f101 kfun:kotlinx.coroutines.internal#shareableInterceptedResumeCancellableWith@kotlin.coroutines.Continuation<0:0>(kotlin.Result<0:0>){0§<kotlin.Any?>} + 433 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/internal/Sharing.kt:59:18)
    at 9   libdrill_agent.dylib                0x000000010a3323d1 kfun:kotlinx.coroutines.DispatchedCoroutine#afterResume(kotlin.Any?){} + 369 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/Builders.common.kt:312:15)
    at 10  libdrill_agent.dylib                0x000000010a32f0f4 kfun:kotlinx.coroutines.AbstractCoroutine#resumeWith(kotlin.Result<1:0>){} + 452 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/AbstractCoroutine.kt:113:9)
    at 11  libdrill_agent.dylib                0x000000010a1cd6a0 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 1440 (/Users/teamcity/buildAgent/work/f01984a9f5203417/runtime/src/main/kotlin/kotlin/coroutines/ContinuationImpl.kt:43:32)
    at 12  libdrill_agent.dylib                0x000000010a3707ce kfun:kotlinx.coroutines.DispatchedTask#run(){} + 2798 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/internal/DispatchedTask.kt:106:71)
    at 13  libdrill_agent.dylib                0x000000010a34711f kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long + 831 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:274:18)
    ... and 9 more common stack frames skipped

OS is MacOS BigSur (11.6.5), Java AdoptOpenJDK 11.0.9, Spring Boot v2.5.1, Spring core version '5.3.8', drill4j agent version v0.7.0. debugagent parameter: -agentpath:/Users/xxx/Downloads/macosX64-0.7.0/libdrill_agent.dylib=drillInstallationDir=/Users/xxx/Downloads/macosX64-0.7.0,adminAddress=localhost:8090,agentId=XXX,buildVersion=0.1.0,logLevel=INFO

I disabled loading of plugins from github (DRILL_PLUGINS_REMOTE_ENABLED=false) because I have network restrictions and this loading failed:

  drill-admin:
    image: 'dockerproxy.aexp.com/drill4j/admin:0.7.0'
    environment:
      - TEST2CODE_PLUGIN_VERSION=0.7.0
      - LOG_LEVEL=INFO
      - DRILL_PLUGINS_REMOTE_ENABLED=false
    ports:
      - 8090:8090
  admin-ui:
    image: 'dockerproxy.aexp.com/drill4j/admin-ui:0.7.1'
    ports:
      - 8091:8080

Logs of drill admin:

2022-05-16 22:42:12.691 [main] INFO  Application - Autoreload is disabled because the development mode is off.
2022-05-16T22:42:21.318594200Z 2022-05-16 22:42:21.317 [main] INFO  jetbrains.exodus.env.EnvironmentImpl - Exodus environment created: work/common
2022-05-16T22:42:26.917316000Z 2022-05-16 22:42:26.913 [DefaultDispatcher-worker-1] INFO  c.e.d.a.plugins.PluginLoaderService - Searching for plugins in paths [/distr/adminStorage]
2022-05-16T22:42:26.921295300Z 2022-05-16 22:42:26.919 [DefaultDispatcher-worker-1] WARN  c.e.d.a.plugins.PluginLoaderService - No plugins found!
2022-05-16T22:42:30.817372100Z 2022-05-16 22:42:30.810 [main] INFO  Application - Responding at http://0.0.0.0:8090
2022-05-16T22:42:46.348172700Z 2022-05-16 22:42:46.347 [DefaultDispatcher-worker-1] INFO  jetbrains.exodus.env.EnvironmentImpl - Exodus environment created: work/agents/XXX
2022-05-16T22:43:45.869163700Z 2022-05-16 22:43:45.868 [eventLoopGroupProxy-3-5] INFO  c.e.d.admin.endpoints.AgentManager - Agent with id 'XXX' was disconnected

Logs of drill admin ui:

--------------------------------------------------------
2022-05-16T22:46:57.577179700Z  docker-compose-wait 2.7.3
2022-05-16T22:46:57.577200700Z ---------------------------
2022-05-16T22:46:57.577216200Z Starting with configuration:
2022-05-16T22:46:57.577235100Z  - Hosts to be waiting for: []
2022-05-16T22:46:57.577251500Z  - Timeout before failure: 30 seconds 
2022-05-16T22:46:57.577267300Z  - TCP connection timeout before retry: 5 seconds 
2022-05-16T22:46:57.577282100Z  - Sleeping time before checking for hosts availability: 0 seconds
2022-05-16T22:46:57.577301200Z  - Sleeping time once all hosts are available: 0 seconds
2022-05-16T22:46:57.577319200Z  - Sleeping time between retries: 1 seconds
2022-05-16T22:46:57.577337600Z --------------------------------------------------------
2022-05-16T22:46:57.577352400Z docker-compose-wait - Everything's fine, the application can now start!
2022-05-16T22:46:57.577370300Z --------------------------------------------------------
2022-05-16T22:47:03.445979800Z 172.18.0.1 - - [16/May/2022:22:47:03 +0000] "GET /ws/plugins/test2code?token=null HTTP/1.1" 404 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"
2022-05-16T22:47:04.968637200Z 172.18.0.1 - - [16/May/2022:22:47:04 +0000] "GET /login HTTP/1.1" 304 0 "http://localhost:8091/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"
2022-05-16T22:47:04.974889800Z 172.18.0.1 - - [16/May/2022:22:47:04 +0000] "GET /ws/drill-admin-socket?token=null HTTP/1.1" 101 69 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"
2022-05-16T22:47:05.033102300Z 172.18.0.1 - - [16/May/2022:22:47:05 +0000] "GET /ws/plugins/test2code?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJBdXRoZW50aWNhdGlvbiIsInJvbGUiOiJhZG1pbiIsImlzcyI6Imh0dHA6Ly9kcmlsbC00LWovIiwiaWQiOjEsImV4cCI6MTY1NzkyNTIyNH0.VZIy-3UTwznFCUQcGJDYcVhFDgMJ4VvvNwAQ13WLBhMV8xfWqkzo_pS_ZLMPfHXxWZuGYjGHgkd2C8rNcMySoA HTTP/1.1" 404 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36"

I suspect the issue may happen because I didn't add plugins. But I haven't found how to add them manually. Could you please advise?

Aliaksei-Tatarynchyk avatar May 16 '22 22:05 Aliaksei-Tatarynchyk