snakeyaml errors are not being reported to users

[mr-c]

Is the disk full again @swzcuroverse?

Error: An unexpected error occurred when parsing the workflow

With https://github.com/emo-bon/pipeline-v5/blob/develop/workflows/gos_wf.cwl

[kinow]

I started the Docker Compose cluster locally using the latest code on master. Tried to import the workflow above, and got:

spring_1_905cde106a10 | 2022-09-22 23:10:49,101 WARN  [http-nio-8080-exec-10] org.commonwl.view.workflow.WorkflowController: url.parsingError WorkflowForm [url=https://github.com/emo-bon/pipeline-v5/blob/develop/workflows/gos_wf.cwl, branch=, path=]
spring_1_905cde106a10 | org.yaml.snakeyaml.parser.ParserException: while parsing a flow mapping
spring_1_905cde106a10 |  in 'reader', line 34, column 19:
spring_1_905cde106a10 |       ncrna_tab_file: {type: File?}
spring_1_905cde106a10 |                       ^
spring_1_905cde106a10 | expected ',' or '}', but got ?
spring_1_905cde106a10 |  in 'reader', line 34, column 30:
spring_1_905cde106a10 |       ncrna_tab_file: {type: File?}
spring_1_905cde106a10 |                                  ^
spring_1_905cde106a10 | 
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.parser.ParserImpl$ParseFlowMappingKey.produce(ParserImpl.java:905)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:171)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:57)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:43)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:136)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:116)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeScalarNode(Composer.java:221)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:190)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:321)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:312)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:293)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:194)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:321)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:312)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:293)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:194)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:321)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:312)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:293)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:194)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.getNode(Composer.java:116)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:142)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:155)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:477)
spring_1_905cde106a10 | 	at org.yaml.snakeyaml.Yaml.load(Yaml.java:418)
spring_1_905cde106a10 | 	at org.commonwl.view.cwl.CWLService.yamlStreamToJson(CWLService.java:622)
spring_1_905cde106a10 | 	at org.commonwl.view.cwl.CWLService.parseWorkflowNative(CWLService.java:179)
spring_1_905cde106a10 | 	at org.commonwl.view.cwl.CWLService.parseWorkflowNative(CWLService.java:252)
spring_1_905cde106a10 | 	at org.commonwl.view.workflow.WorkflowService.createQueuedWorkflow(WorkflowService.java:349)
spring_1_905cde106a10 | 	at org.commonwl.view.workflow.WorkflowController.createWorkflow(WorkflowController.java:140)
spring_1_905cde106a10 | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
spring_1_905cde106a10 | 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
spring_1_905cde106a10 | 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
spring_1_905cde106a10 | 	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
spring_1_905cde106a10 | 	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
spring_1_905cde106a10 | 	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
spring_1_905cde106a10 | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
spring_1_905cde106a10 | 	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
spring_1_905cde106a10 | 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
spring_1_905cde106a10 | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
spring_1_905cde106a10 | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
spring_1_905cde106a10 | 	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
spring_1_905cde106a10 | 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
spring_1_905cde106a10 | 	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
spring_1_905cde106a10 | 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
spring_1_905cde106a10 | 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
spring_1_905cde106a10 | 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
spring_1_905cde106a10 | 	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
spring_1_905cde106a10 | 	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
spring_1_905cde106a10 | 	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
spring_1_905cde106a10 | 	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
spring_1_905cde106a10 | 	at java.base/java.lang.Thread.run(Thread.java:833)

[kinow]

Failed to parse on http://www.yamllint.com/ as well.

[kinow]

Looks to be an issue with the optional types, like File? inside the brackets. Rewriting obj {type: File?} to

obj:
  type: File?

everywhere in that workflow, and re-running YAMLint it passed validation. After some ~20 minutes I think, I got the graph for the workflow edited from my fork/branch:

https://view.commonwl.org/workflows/github.com/kinow/pipeline-v5/blob/eosc-life-gos/workflows/gos_wf.cwl

[kinow]

(might be something to add to the YAML topic in our user guide?)

[mr-c]

Thanks @kinow for the investigation.

ruaml.yaml accepts this YAML, alas.

[mr-c]

Possibly the issue is that CWL is YAML 1.2, but snakeyaml is for YAML 1.1 ; maybe we need to switch to snakeyaml-engine or eo-yaml

https://bitbucket.org/snakeyaml/snakeyaml-engine/src/master/

https://github.com/decorators-squad/eo-yaml

[kinow]

Ah, that'd make sense. Shouldn't be too hard to test either of these libraries if they are published to Maven central repo (Java's PYPI equivalent I think).