elyra
elyra copied to clipboard
elyra-pipeline validate CLI command fails for pipelines with parameters and default properties
Describe the issue
Running elyra-pipeline validate
and by extension elyra-pipeline export
fails silently with pipelines that contain both pipeline parameters and default properties.
To Reproduce Steps to reproduce the behavior:
- Create a simple Elyra pipeline with a single node.
- Add a pipeline parameter in the
Pipeline Parameters
configuration tab. - Open the
Pipeline Properties
configuration tab. - Save the pipeline, e.g. as
test.pipeline
. - Validate the pipeline through the CLI:
elyra-pipeline validate test.pipeline
.
Screenshots or log output The following error message without details is printed after running the above validation command:
Log Output
(app-root) (app-root) elyra-pipeline validate test.pipeline──────────────────────────────────────────────────────────────── Elyra Pipeline Validation ────────────────────────────────────────────────────────────────
Validating pipeline... Error: Pipeline validation FAILED.
Expected behavior Given that pipelines with parameters and default properties are valid in general, the validation command should resolve without any errors. In case of actual issues within the pipeline file, there should be detailed information about these issues.
Deployment information Describe what you've deployed and how:
- Elyra version: 3.15.0
- Installation source: quay.io/opendatahub/workbench-images:cuda-jupyter-tensorflow-ubi9-python-3.9
- Deployment type: Red Hat OpenShift Data Science 1.33.0
- Operating system: linux
Example pipeline:
{
"doc_type": "pipeline",
"version": "3.0",
"json_schema": "http://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json",
"id": "elyra-auto-generated-pipeline",
"primary_pipeline": "primary",
"pipelines": [
{
"id": "primary",
"nodes": [
{
"id": "54d57acc-4ad0-410e-8d79-268cd52a45e9",
"type": "execution_node",
"op": "execute-python-node",
"app_data": {
"component_parameters": {
"pipeline_parameters": [
"param"
],
"dependencies": [],
"include_subdirectories": false,
"outputs": [],
"env_vars": [],
"kubernetes_pod_annotations": [],
"kubernetes_pod_labels": [],
"kubernetes_secrets": [],
"kubernetes_shared_mem_size": {},
"kubernetes_tolerations": [],
"mounted_volumes": [],
"filename": "untitled.py",
"runtime_image": "quay.io/modh/runtime-images@sha256:7dd23e58291cad7a0ab4a8e04bda06492f2c027eb33b226358380db58dcdd60b"
},
"label": "",
"ui_data": {
"label": "untitled.py",
"image": "/notebook/fraud-detection/tensorflow/static/elyra/python.svg",
"x_pos": 505,
"y_pos": 210,
"description": "Run Python script"
}
},
"inputs": [
{
"id": "inPort",
"app_data": {
"ui_data": {
"cardinality": {
"min": 0,
"max": -1
},
"label": "Input Port"
}
}
}
],
"outputs": [
{
"id": "outPort",
"app_data": {
"ui_data": {
"cardinality": {
"min": 0,
"max": -1
},
"label": "Output Port"
}
}
}
]
}
],
"app_data": {
"ui_data": {
"comments": []
},
"version": 8,
"runtime_type": "KUBEFLOW_PIPELINES",
"properties": {
"name": "untitled",
"runtime": "Data Science Pipelines",
"pipeline_parameters": [
{
"name": "param",
"description": "",
"default_value": {
"type": "String",
"value": "val"
},
"required": false
}
],
"pipeline_defaults": {
"kubernetes_shared_mem_size": {},
"kubernetes_pod_labels": [],
"mounted_volumes": [],
"kubernetes_pod_annotations": [],
"kubernetes_tolerations": [],
"kubernetes_secrets": [],
"env_vars": []
}
}
},
"runtime_ref": ""
}
],
"schemas": []
}
Validation through the CLI fails. Remove either pipeline_parameters
or pipeline_defaults
, and validation succeeds.