charts
charts copied to clipboard
[artifactory] - Can't start service after upgrade from 7.18.6 to 7.21.8
Is this a request for help?: yes Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG Version of Helm and Kubernetes: Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.13", GitCommit:"30d651da517185653e34e7ab99a792be6a3d9495", GitTreeState:"clean", BuildDate:"2020-10-15T00:59:17Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"} version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"} Which chart: Artifactory
What happened: After upgrade the service is in crash loopback
2021-07-19T19:01:08.165Z [jfrou] [INFO ] [3bd24794e59dbb4a] [bootstrap.go:76 ] [main ] - Router (jfrou) service initialization started. Version: 7.21.5-1 Revision: caee73ee2d273301193e39a341670f5587022613 PID: 3635 Home: /opt/jfrog/artifactory 2021-07-19T19:01:08.166Z [jfrou] [INFO ] [3bd24794e59dbb4a] [bootstrap.go:79 ] [main ] - JFrog Router IP: 172.20.2.72 JF_METADATA_ACCESSCLIENT_URL: http://localhost:8081/access metadata started. PID: 3834 [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [TRACE] Config key not set for aws secret (metadata.database.secretsManagerAlias) Starting event... event not running. Proceed to start it up. event started. PID: 3992 [INFO ] JFrog Event (jfevt) service initialization started. Version: 7.21.5 (revision: caee73ee2d, build date: 2021-07-11T12:43:46Z) PID: 3992 Home: /opt/jfrog/artifactory [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml Starting frontend... frontend not running. Proceed to start it up. frontend started. PID: 4153 2021-07-19T19:01:09.000Z [shell] [INFO ] [] [installerCommon.sh:1188 ] [main] - Redirection is set to false. Skipping catalina log redirection NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 2021-07-19T19:01:10.371L [tomct] [INFO ] [ ] [org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandler ["http-nio-8081"] 2021-07-19T19:01:10.378Z [jffe ] [INFO ] [ ] [ ] [main ] - frontend (jffe) service initialization started. Version: 1.21.4 Revision: 10000004 PID: 4349 Home: /opt/jfrog/artifactory 2021-07-19T19:01:10.380Z [jffe ] [INFO ] [ ] [ ] [main ] - attempting pinging artifactory for 180 retires and 1.0s interval for total of 3 minutes 2021-07-19T19:01:10.396L [tomct] [INFO ] [ ] [org.apache.tomcat.util.net.NioSelectorPool] [org.apache.tomcat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read 2021-07-19T19:01:10.442L [tomct] [INFO ] [ ] [org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandler ["http-nio-127.0.0.1-8091"] 2021-07-19T19:01:10.443L [tomct] [INFO ] [ ] [org.apache.tomcat.util.net.NioSelectorPool] [org.apache.tomcat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read 2021-07-19T19:01:10.445L [tomct] [INFO ] [ ] [org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandler ["http-nio-127.0.0.1-8040"] 2021-07-19T19:01:10.446L [tomct] [INFO ] [ ] [org.apache.tomcat.util.net.NioSelectorPool] [org.apache.tomcat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read 2021-07-19T19:01:10.455L [tomct] [INFO ] [ ] [org.apache.catalina.core.StandardService] [org.apache.catalina.core.StandardService startInternal] - Starting service [Catalina] 2021-07-19T19:01:10.456L [tomct] [INFO ] [ ] [org.apache.catalina.core.StandardEngine] [org.apache.catalina.core.StandardEngine startInternal] - Starting Servlet engine: [Apache Tomcat/8.5.66] 2021-07-19T19:01:10.496L [tomct] [INFO ] [ ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deploying deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/access.xml] 2021-07-19T19:01:10.496L [tomct] [INFO ] [ ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deploying deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml] 2021-07-19T19:01:10.556L [tomct] [WARNING] [ ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase [/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/access.war] inside the host appBase has been specified, and will be ignored 2021-07-19T19:01:10.556L [tomct] [WARNING] [ ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase [/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/artifactory.war] inside the host appBase has been specified, and will be ignored 2021-07-19T19:01:13.175Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 5 seconds with 2m0s timeout 2021-07-19T19:01:13.469Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 5 seconds with 2m0s timeout [init] 2021-07-19T19:01:13.500Z [jfrt ] [INFO ] [10abdf52b7de7daa] [o.a.c.h.HaNodeProperties:65 ] [ocalhost-startStop-1] - Artifactory is running in non-clustered mode. 2021-07-19T19:01:13.545Z [jfrt ] [INFO ] [10abdf52b7de7daa] [tifactoryHomeConfigListener:85] [ocalhost-startStop-1] - Resolved Home: '/opt/jfrog/artifactory 2021-07-19T19:01:13.761Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 5 seconds with 2m0s timeout [startup] 2021-07-19T19:01:13.922Z [jfac ] [INFO ] [9d0282986f5e7389] [licationContextInitializer:162] [ocalhost-startStop-2] - Access (jfac) service initialization started. Version: 7.21.5 Revision: 72105900 PID: 4329 Home: /opt/jfrog/artifactory 2021-07-19T19:01:14.000Z [jfac ] [INFO ] [9d0282986f5e7389] [o.j.a.AccessApplication:55 ] [ocalhost-startStop-2] - Starting AccessApplication v7.21.5 using Java 11.0.10 on artifactory-0 with PID 4329 (/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/access/WEB-INF/lib/access-application-7.21.5.jar started by root in /var/opt/jfrog/artifactory) 2021-07-19T19:01:14.001Z [jfac ] [INFO ] [9d0282986f5e7389] [o.j.a.AccessApplication:679 ] [ocalhost-startStop-2] - The following profiles are active: production 2021-07-19T19:01:14.062Z [jfrt ] [INFO ] [ ] [o.j.c.w.FileWatcher:147 ] [Thread-9 ] - Starting watch of folder configurations 2021-07-19T19:01:14.215Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.BasicConfigurationManager:186] [ocalhost-startStop-1] - Artifactory (jfrt) service initialization started. Version: 7.21.8 Revision: 72108900 PID: 4329 Home: /opt/jfrog/artifactory 2021-07-19T19:01:14.588Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.c.m.ConverterManagerImpl:212] [ocalhost-startStop-1] - Triggering PRE_INIT conversion, from 7.18.6 to 7.21.8 2021-07-19T19:01:14.588Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.c.m.ConverterManagerImpl:215] [ocalhost-startStop-1] - Finished PRE_INIT conversion, current version is: 7.21.8 2021-07-19T19:01:14.588Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.i.DbInitializationManager:49] [ocalhost-startStop-1] - Initializing DB Schema initialization manager 2021-07-19T19:01:14.590Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.i.DbInitializationManager:177] [ocalhost-startStop-1] - Database: PostgreSQL 12.4. Driver: PostgreSQL JDBC Driver 42.2.19 Pool: postgresql 2021-07-19T19:01:15.153Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.i.DbInitializationManager:53] [ocalhost-startStop-1] - DB Schema initialization manager initialized 2021-07-19T19:01:15.163Z [jfrt ] [INFO ] [10abdf52b7de7daa] [SchemaInitializationManager:48] [ocalhost-startStop-1] - Initializing Post-DB initialization manager 2021-07-19T19:01:15.203Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.c.m.ConverterManagerImpl:212] [ocalhost-startStop-1] - Triggering MARKERS conversion, from 7.18.6 to 7.21.8 2021-07-19T19:01:15.210Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.c.m.ConverterManagerImpl:215] [ocalhost-startStop-1] - Finished MARKERS conversion, current version is: 7.21.8 2021-07-19T19:01:15.649Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.c.m.ConverterManagerImpl:212] [ocalhost-startStop-1] - Triggering HOME_FILES conversion, from 7.18.6 to 7.21.8 2021-07-19T19:01:15.789Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.l.c.LogbackConverterUtils:102] [ocalhost-startStop-1] - appender REQUEST_OUT already exists in logback.xml, skipping conversion 2021-07-19T19:01:15.791Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.l.c.LogbackConverterUtils:102] [ocalhost-startStop-1] - logger org.artifactory.traffic.OutboundRequestLogger already exists in logback.xml, skipping conversion 2021-07-19T19:01:15.853Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.l.c.LogbackConverterUtils:102] [ocalhost-startStop-1] - appender REQUEST_OUT already exists in logback.xml, skipping conversion 2021-07-19T19:01:15.856Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.l.c.LogbackConverterUtils:102] [ocalhost-startStop-1] - logger org.artifactory.traffic.OutboundRequestLogger already exists in logback.xml, skipping conversion 2021-07-19T19:01:15.881Z [jfrt ] [INFO ] [10abdf52b7de7daa] [o.a.l.v.LoggingVersion:140 ] [ocalhost-startStop-1] - Ending database conversion from 7.18.6 2021-07-19T19:01:15.896Z [jfrt ] [INFO ] [10abdf52b7de7daa] [d.c.m.ConverterManagerImpl:215] [ocalhost-startStop-1] - Finished HOME_FILES conversion, current version is: 7.21.8 2021-07-19T19:01:15.897Z [jfrt ] [INFO ] [10abdf52b7de7daa] [.c.ConfigurationManagerImpl:97] [ocalhost-startStop-1] - Starting file sync 2021-07-19T19:01:16.072Z [jfrt ] [INFO ] [10abdf52b7de7daa] [SchemaInitializationManager:51] [ocalhost-startStop-1] - Post-DB initialization manager initialized 2021-07-19T19:01:16.760Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [ctoryContextConfigListener:330] [art-init ] - _ _ __ _ _____ /\ | | (_)/ _| | | | __ \ / \ _ __| |_ _| |_ __ _ ___| |_ ___ _ __ _ _ | |__) | __ ___ / /\ \ | '__| __| | _/ _
|/ | / _ | '| | | | | / '/ _
/ ____ | | | || | || (_| | (| || () | | | || | | | | | | () |
// __| _||| _,|___|__/|| _, | || || _/
Version: 7.21.8 / |
Revision: 72108900 |_/
Artifactory Home: '/opt/jfrog/artifactory'
Node ID: 'artifactory-0'
2021-07-19T19:01:16.926L [tomct] [INFO ] [ ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deployment of deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml] has finished in [6,428] ms
2021-07-19T19:01:17.055Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [ifactoryApplicationContext:560] [art-init ] - Artifactory application context set to NOT READY by refresh
2021-07-19T19:01:18.180Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 10 seconds with 2m0s timeout
2021-07-19T19:01:18.478Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 10 seconds with 2m0s timeout [init]
2021-07-19T19:01:18.766Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 10 seconds with 2m0s timeout [startup]
2021-07-19T19:01:19.128Z [jfac ] [INFO ] [9d0282986f5e7389] [s.d.u.AccessJdbcHelperImpl:140] [ocalhost-startStop-2] - Database: PostgreSQL 12.4. Driver: PostgreSQL JDBC Driver 42.2.19
2021-07-19T19:01:19.442Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 10
2021-07-19T19:01:19.445Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 10 failed with code : ECONNREFUSED
2021-07-19T19:01:23.186Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 15 seconds with 2m0s timeout
2021-07-19T19:01:23.482Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 15 seconds with 2m0s timeout [init]
2021-07-19T19:01:23.770Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 15 seconds with 2m0s timeout [startup]
2021-07-19T19:01:28.191Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 20 seconds with 2m0s timeout
2021-07-19T19:01:28.487Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 20 seconds with 2m0s timeout [init]
2021-07-19T19:01:28.774Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 20 seconds with 2m0s timeout [startup]
2021-07-19T19:01:28.912Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [ritiesStorageServiceFactory:25] [art-init ] - Initializing DB-based Priorities Storage Service
2021-07-19T19:01:29.482Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 20
2021-07-19T19:01:29.485Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 20 failed with code : ECONNREFUSED
WARN freemarker.log.LoggerFactory: Logger library was already set earlier to "auto"; change to "none" won't effect loggers created earlier.
2021-07-19T19:01:32.899Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [HaRoleManagerServiceFactory:27] [art-init ] - Initializing Task Affinity HA role manager
2021-07-19T19:01:33.199Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 25 seconds with 2m0s timeout
2021-07-19T19:01:33.492Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 25 seconds with 2m0s timeout [init]
2021-07-19T19:01:33.659Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [actorySchedulerFactoryBean:727] [art-init ] - Starting Quartz Scheduler now
2021-07-19T19:01:33.739Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [ifactoryApplicationContext:281] [art-init ] - Artifactory context starting up 73 Spring Beans...
2021-07-19T19:01:33.777Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 25 seconds with 2m0s timeout [startup]
2021-07-19T19:01:34.701Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.a.s.a.AccessServiceImpl:525 ] [art-init ] - Initialized new service id: jfrt@01e5z1q1cv0mvk1tby0d6c0de4
2021-07-19T19:01:34.756Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [oryAccessClientConfigStore:528] [art-init ] - Using Access Server URL: http://localhost:8046/access source: System Config
2021-07-19T19:01:38.205Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 30 seconds with 2m0s timeout
2021-07-19T19:01:38.496Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 30 seconds with 2m0s timeout [init]
2021-07-19T19:01:38.781Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 30 seconds with 2m0s timeout [startup]
2021-07-19T19:01:39.527Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 30
2021-07-19T19:01:39.531Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 30 failed with code : ECONNREFUSED
2021-07-19T19:01:39.960Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:01:43.213Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 35 seconds with 2m0s timeout
2021-07-19T19:01:43.499Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 35 seconds with 2m0s timeout [init]
2021-07-19T19:01:43.786Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 35 seconds with 2m0s timeout [startup]
2021-07-19T19:01:44.972Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:01:48.217Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 40 seconds with 2m0s timeout
2021-07-19T19:01:48.505Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 40 seconds with 2m0s timeout [init]
2021-07-19T19:01:48.789Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 40 seconds with 2m0s timeout [startup]
2021-07-19T19:01:49.568Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 40
2021-07-19T19:01:49.572Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 40 failed with code : ECONNREFUSED
2021-07-19T19:01:49.982Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:01:53.222Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 45 seconds with 2m0s timeout
2021-07-19T19:01:53.508Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 45 seconds with 2m0s timeout [init]
2021-07-19T19:01:53.792Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 45 seconds with 2m0s timeout [startup]
2021-07-19T19:01:54.994Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:01:58.228Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 50 seconds with 2m0s timeout
2021-07-19T19:01:58.513Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 50 seconds with 2m0s timeout [init]
2021-07-19T19:01:58.796Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 50 seconds with 2m0s timeout [startup]
2021-07-19T19:01:59.609Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 50
2021-07-19T19:01:59.611Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 50 failed with code : ECONNREFUSED
2021-07-19T19:02:00.006Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:03.237Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 55 seconds with 2m0s timeout
2021-07-19T19:02:03.516Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 55 seconds with 2m0s timeout [init]
2021-07-19T19:02:03.800Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 55 seconds with 2m0s timeout [startup]
2021-07-19T19:02:05.019Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:08.243Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 60 seconds with 2m0s timeout
2021-07-19T19:02:08.520Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 60 seconds with 2m0s timeout [init]
2021-07-19T19:02:08.805Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 60 seconds with 2m0s timeout [startup]
2021-07-19T19:02:09.647Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 60
2021-07-19T19:02:09.649Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 60 failed with code : ECONNREFUSED
2021-07-19T19:02:10.029Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:13.250Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 65 seconds with 2m0s timeout
2021-07-19T19:02:13.526Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 65 seconds with 2m0s timeout [init]
2021-07-19T19:02:13.808Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 65 seconds with 2m0s timeout [startup]
2021-07-19T19:02:15.041Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:18.255Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 70 seconds with 2m0s timeout
2021-07-19T19:02:18.533Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 70 seconds with 2m0s timeout [init]
2021-07-19T19:02:18.813Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 70 seconds with 2m0s timeout [startup]
2021-07-19T19:02:19.686Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 70
2021-07-19T19:02:19.688Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 70 failed with code : ECONNREFUSED
2021-07-19T19:02:20.068Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:23.261Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 75 seconds with 2m0s timeout
2021-07-19T19:02:23.537Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 75 seconds with 2m0s timeout [init]
2021-07-19T19:02:23.815Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 75 seconds with 2m0s timeout [startup]
2021-07-19T19:02:25.081Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:28.266Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 80 seconds with 2m0s timeout
2021-07-19T19:02:28.546Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 80 seconds with 2m0s timeout [init]
2021-07-19T19:02:28.818Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 80 seconds with 2m0s timeout [startup]
2021-07-19T19:02:29.729Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 80
2021-07-19T19:02:29.733Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 80 failed with code : ECONNREFUSED
2021-07-19T19:02:30.092Z [jfrt ] [INFO ] [31f65b22cfacb5ea] [o.j.s.c.KeyUtils:108 ] [art-init ] - Cluster join: Join key is missing. Pending for 5 seconds with 60 seconds timeout
2021-07-19T19:02:33.273Z [jfrou] [INFO ] [3bd24794e59dbb4a] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 85 seconds with 2m0s timeout
2021-07-19T19:02:33.550Z [jfmd ] [INFO ] [5c092419d70690de] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 85 seconds with 2m0s timeout [init]
2021-07-19T19:02:33.821Z [jfevt] [INFO ] [648b937de928e982] [security_keys.go:163 ] [main ] - Cluster join: Join key is missing. Pending for 85 seconds with 2m0s timeout [startup]
2021-07-19T19:02:35.139Z [jfrt ] [ERROR] [31f65b22cfacb5ea] [ctoryContextConfigListener:128] [art-init ] - Application could not be initialized: Cluster join: Failed resolving join key; Missing join key
java.lang.reflect.InvocationTargetException: null
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.artifactory.lifecycle.webapp.servlet.ArtifactoryContextConfigListener.configure(ArtifactoryContextConfigListener.java:270)
at org.artifactory.lifecycle.webapp.servlet.ArtifactoryContextConfigListener$1.run(ArtifactoryContextConfigListener.java:124)
Caused by: org.springframework.beans.factory.BeanInitializationException: Failed to initialize bean 'org.artifactory.security.access.AccessService'.; nested exception is java.lang.IllegalStateException: Cluster join: Failed resolving join key; Missing join key
at org.artifactory.spring.ArtifactoryApplicationContext.initReloadableBeans(ArtifactoryApplicationContext.java:302)
at org.artifactory.spring.ArtifactoryApplicationContext.refresh(ArtifactoryApplicationContext.java:284)
at org.artifactory.spring.ArtifactoryApplicationContext.feature-policy
module instead. at ../../../../opt/jfrog/artifactory/app/frontend/bin/server/dist/bundle.js:13530:24
2021-07-19T19:04:11.412Z [jffe ] [INFO ] [ ] [ ] [main ] - Waiting for Express to close all connections
2021-07-19T19:04:12.232Z [jffe ] [INFO ] [ ] [ ] [main ] - frontend (jffe) service initialization started. Version: 1.21.4 Revision: 10000004 PID: 4390 Home: /opt/jfrog/artifactory
2021-07-19T19:04:12.241Z [jffe ] [INFO ] [ ] [ ] [main ] - attempting pinging artifactory for 180 retires and 1.0s interval for total of 3 minutes
2021-07-19T19:04:21.312Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 10
2021-07-19T19:04:21.314Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 10 failed with code : ECONNREFUSED
2021-07-19T19:04:31.352Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 20
2021-07-19T19:04:31.355Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 20 failed with code : ECONNREFUSED
2021-07-19T19:04:41.401Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 30
2021-07-19T19:04:41.405Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 30 failed with code : ECONNREFUSED
2021-07-19T19:04:51.438Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 40
2021-07-19T19:04:51.442Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 40 failed with code : ECONNREFUSED
`
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know: I copied manually the join.key cp /var/opt/jfrog/artifactory/bootstrap/access/etc/security/join.key var/etc/security/
But the result is the same.
The join.key it's ok in the helm value.
@rdcarrera Can you please provide the chart versions used and the custom helm values. Also please provide the full logs.
I also have the same issue with Chart Version 107.21.7 and AppVersion 7.21.3. Are there already any insights?
I'm seeing a similar looking log file to this - did anyone have any luck?
I'm using 107.25.7 / 7.25.7 of the ha chart.
I did jump into my container and looked under var/etc/security and I notice the master key is the same as the decrypted value of the secret that created it, but the join key is not, and I'm not sure if that is to be expected.
When using helm, what I would advise against this error is checking your app logs (like artifactory-service.log) because what we investigated today is that the join.key may work only at bootstrap (is then consumed or something) and when your service doesn't start (like database issue) what you will see is a lot of logs about join.key but it could not be the root cause!
@chukka your POV on this?
@kaychamo you are right, there are few services(metadata,frontend, event, etc.,) inside artifactory which tries to connect to access via joinKey through router
from above logs , jfmd metadata and jffe (frontend) services failed to connect.
Also, there is a provision to change joinKey via values.yaml