kestra
kestra copied to clipboard
GitHub Action and CLI Validate marking 1 flow as invalid when it's valid
Describe the issue
Execute:
./run kestra:cli 1 flow validate kestra_flows/automations/bigquery --server=https://us.kestra.cloud --api-token=<API-TOKEN> --tenant=<TENANT-DEV-ID>
Result:
✘ - x.automations.bigquery.bigquery_dedupper
Unable to parse flows due to the following error:
- tasks[0].taskRunner.region: must not be null
Workflow YAML:
...
tasks:
- id: run_deduplication_python_command
type: io.kestra.plugin.scripts.python.Commands
beforeCommands:
- export PYTHONPATH=src
- export GCP_PROJECT_ID={{ secret('GCP_PROJECT_ID') }}
containerImage: <valid-image-uri>
commands:
- >
python src/automations/bigquery/dedupper/main.py
...
warningOnStdErr: false
taskRunner:
type: io.kestra.plugin.ee.gcp.runner.Batch
waitUntilCompletion: 18000
machineType: e2-micro
computeResource:
cpu: "2000"
memory: "1024"
...
Plugin Defaults in root namespace:
- type: io.kestra.plugin.scripts.python.Commands
forced: false
values:
taskRunner:
type: io.kestra.plugin.ee.gcp.runner.Batch
bucket: "{{ secret('GCS_BUCKET') }}"
projectId: "{{ secret('GCP_PROJECT_ID') }}"
region: <gcp valid region string here>
serviceAccount: "{{ secret('GCP_SA_JSON') }}"
I think it does not validate well when plugin_defaults is present for ee.batch region.
Let me know your all's thoughts, or if you need me to do anything debug-wise.
Thank you team 🙏 🚀
Environment
- Kestra Version: 0.18.6
- Operating System (OS/Docker/Kubernetes): Kestra Cloud (validate command ran locally in docker container with CLI command 0.18.6)
- Java Version (if you don't run kestra in Docker): n/a