gocd icon indicating copy to clipboard operation
gocd copied to clipboard
verified publisher icon gocd

Creating a pipeline through api that references a config repo pipeline will crash the server after restart

Open tinaboyce opened this issue 1 year ago • 1 comments

Issue Type
  • Bug Report
Summary

There are two pipeline created:

  • parent, created through config repo
  • ref-cfg-pipeline, created through api to reference parent

In the UI, no pipeline would be able to reference parent due to validation. However, the API does not have such validation and would result in a 200 OK success response.

After an API call, the gocd ui would continue to "work" until a server restart, which then error out with "Error: Pipeline 'parent' does not exist. It is used from pipeline 'ref-cfg-pipeline'.". I have attached the logs of the final few moments before the server exits.

Environment

Using Docker image: gocd/gocd-server:v23.5.0

Basic environment details
  • Go Version: 23.5.0-18179
  • JAVA Version: 17.0.9
  • OS: Linux(6.6.22-linuxkit)
Additional Environment Details
Steps to Reproduce
  1. Deploy a pipeline through the config repo named parent
  2. Create a POST /go/api/admin/pipelines with the following request body after the steps. The API response is 200 OK
  3. Restart the server

Request body:

{
    "group": "test",
    "pipeline": {
        "label_template": "${COUNT}",
        "lock_behavior": "none",
        "name": "ref-cfg-pipeline",
        "materials": [
            {
                "attributes": {
                    "url": "*snip*",
                    "branch": "master",
                    "shallow_clone": false,
                    "auto_update": true,
                    "username": "*snip*",
                    "encrypted_password": "*snip*",
                    "name": "git-config"
                },
                "type": "git"
            },
            {
                "attributes": {
                    "pipeline": "parent",
                    "stage": "test",
                    "ignore_for_scheduling": false,
                    "name": "parent"
                },
                "type": "dependency"
            }
        ],
        "stages": [
            {
                "name": "test",
                "fetch_materials": true,
                "clean_working_directory": false,
                "never_cleanup_artifacts": false,
                "approval": {
                    "type": "success",
                    "authorization": {
                        "roles": [],
                        "users": []
                    }
                },
                "environment_variables": [],
                "jobs": [
                    {
                        "name": "ls",
                        "run_instance_count": null,
                        "timeout": 10,
                        "environment_variables": [],
                        "resources": [],
                        "artifacts": [],
                        "tasks": [
                            {
                                "attributes": {
                                    "command": "ls",
                                    "timeout": 0.0,
                                    "arguments": [],
                                    "run_if": [
                                        "passed"
                                    ]
                                },
                                "type": "exec"
                            }
                        ]
                    }
                ]
            }
        ]
    }
}
Expected Results

The API should reject any API call that would result in a new pipeline resource that would reference the config pipeline.

Actual Results

Server cannot start after a restart. See log snippets.

Possible Fix

Update API validation to match UIs.

Log snippets
gocd-server        | jvm 1    | 2024-08-09 12:21:15,246 WARN  [WrapperJarAppMain] GoConfigMigrator:127 - The versioned config file could be invalid or migrating the versioned config resulted in an invalid configuration
gocd-server        | jvm 1    | com.thoughtworks.go.config.exceptions.GoConfigInvalidException: Pipeline 'parent' does not exist. It is used from pipeline 'ref-cfg-pipeline'.
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.MagicalGoConfigXmlLoader.validateCruiseConfig(MagicalGoConfigXmlLoader.java:133)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.MagicalGoConfigXmlLoader.preprocessAndValidate(MagicalGoConfigXmlLoader.java:122)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.FullConfigSaveFlow.preprocessAndValidate(FullConfigSaveFlow.java:97)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.FullConfigSaveNormalFlow.execute(FullConfigSaveNormalFlow.java:59)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.GoConfigMigrator.upgradeConfigFile(GoConfigMigrator.java:111)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.GoConfigMigrator.upgradeVersionedConfigFile(GoConfigMigrator.java:125)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.GoConfigMigrator.upgrade(GoConfigMigrator.java:100)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.GoConfigMigrator.migrate(GoConfigMigrator.java:88)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.config.CachedGoConfig.upgradeConfig(CachedGoConfig.java:147)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.initializers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:112)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.initializers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:49)
gocd-server        | jvm 1    | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
gocd-server        | jvm 1    | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
gocd-server        | jvm 1    | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:883)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545)
gocd-server        | jvm 1    | 	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
gocd-server        | jvm 1    | 	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
gocd-server        | jvm 1    | 	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:40)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:516)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:151)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.JettyServer.startHandlers(JettyServer.java:174)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.JettyServer.start(JettyServer.java:130)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.GoServer.startServer(GoServer.java:56)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.GoServer.go(GoServer.java:48)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.util.GoLauncher.main(GoLauncher.java:42)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at com.thoughtworks.gocd.Boot.run(Boot.java:90)
gocd-server        | jvm 1    | 	at com.thoughtworks.gocd.Boot.main(Boot.java:56)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at org.tanukisoftware.wrapper.WrapperJarApp.run(WrapperJarApp.java:451)
gocd-server        | jvm 1    | 	at java.base/java.lang.Thread.run(Unknown Source)
gocd-server        | jvm 1    | There are errors in the Cruise config file.  Please read the error message and correct the errors.
gocd-server        | jvm 1    | Once fixed, please restart GoCD.
gocd-server        | jvm 1    | Error: Pipeline 'parent' does not exist. It is used from pipeline 'ref-cfg-pipeline'.
gocd-server        | jvm 1    | 2024-08-09 12:21:15,248 ERROR [WrapperJarAppMain] GoConfigMigrator:65 - There are errors in the Cruise config file.  Please read the error message and correct the errors.
gocd-server        | jvm 1    | Once fixed, please restart GoCD.
gocd-server        | jvm 1    | Error: Pipeline 'parent' does not exist. It is used from pipeline 'ref-cfg-pipeline'.
gocd-server        | jvm 1    | ERROR: Failed to start GoCD server. Please check the logs.
gocd-server        | jvm 1    | java.lang.RuntimeException: Failed to start GoCD server.
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.GoServer.startServer(GoServer.java:61)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.GoServer.go(GoServer.java:48)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.util.GoLauncher.main(GoLauncher.java:42)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at com.thoughtworks.gocd.Boot.run(Boot.java:90)
gocd-server        | jvm 1    | 	at com.thoughtworks.gocd.Boot.main(Boot.java:56)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
gocd-server        | jvm 1    | 	at org.tanukisoftware.wrapper.WrapperJarApp.run(WrapperJarApp.java:451)
gocd-server        | jvm 1    | 	at java.base/java.lang.Thread.run(Unknown Source)
gocd-server        | jvm 1    | Caused by: java.lang.RuntimeException: java.lang.NullPointerException
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.initializers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:156)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.initializers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:49)
gocd-server        | jvm 1    | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
gocd-server        | jvm 1    | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
gocd-server        | jvm 1    | 	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:883)
gocd-server        | jvm 1    | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:545)
gocd-server        | jvm 1    | 	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
gocd-server        | jvm 1    | 	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
gocd-server        | jvm 1    | 	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1049)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:624)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:984)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:740)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:392)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1304)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:901)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:306)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:532)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:40)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:516)
gocd-server        | jvm 1    | 	at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:151)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.JettyServer.startHandlers(JettyServer.java:174)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.JettyServer.start(JettyServer.java:130)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.GoServer.startServer(GoServer.java:56)
gocd-server        | jvm 1    | 	... 14 more
gocd-server        | jvm 1    | Caused by: java.lang.NullPointerException
gocd-server        | jvm 1    | 	at java.base/java.io.File.<init>(Unknown Source)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.service.GoConfigService.artifactsDir(GoConfigService.java:268)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.service.GoConfigService.initialize(GoConfigService.java:134)
gocd-server        | jvm 1    | 	at com.thoughtworks.go.server.initializers.ApplicationInitializer.onApplicationEvent(ApplicationInitializer.java:114)
gocd-server        | jvm 1    | 	... 42 more
gocd-server        | jvm 1    | 2024-08-09 12:21:15,927  INFO [goPluginLocationMonitor] p.c.g.a.p.c.g.a.p.PasswordFilePlugin:72 [plugin-cd.go.authentication.passwordfile] - Loading plugin null version 2.1.2-210
gocd-server        | jvm 1    | 2024-08-09 12:21:15,960  WARN [goPluginLocationMonitor] p.c.g.a.p.c.g.a.p.PasswordFilePlugin:97 [plugin-cd.go.authentication.passwordfile] - Request go.plugin-settings.get-configuration is not supported by plugin.
gocd-server        | wrapper  | <-- Wrapper Stopped

-->

Code snippets/Screenshots

Screenshot of the pipeline created through API, referencing the config repo pipeline parent, and showing that the UI only allows parent-ui Screenshot 2024-08-09 at 10 05 04 PM

Any other info

None I could think of.

tinaboyce avatar Aug 09 '24 12:08 tinaboyce

This issue has been automatically marked as stale because it has not had activity in the last 90 days. If you can still reproduce this error on the master branch using local development environment or on the latest GoCD Release, please reply with all of the information you have about it in order to keep the issue open. Thank you for all your contributions.

stale[bot] avatar Apr 26 '25 06:04 stale[bot]