run_workflow_template only passes variables when prompt on launch selected in Workflow Job Template

[adworjan]

Please confirm the following

• [X] I agree to follow this project's code of conduct.
• [X] I have checked the current issues for duplicates.
• [X] I understand that ansible-rulebook is open source software provided for free and that I might not receive a timely response.

Bug Summary

When using the run_workflow_template and a survey, the workflow job only receives the variables when prompt on launch is also selected for extra vars. This is not an issue when directly calling the API, but is an issue when using the run_workflow_template action

Environment

ansible-rulebook-1.0.3 Ansible Automation Platform Controller 4.4.8

Steps to reproduce

1. Create a workflow job template
2. Create and enable a survey (do NOT check prompt on launch for extra variables in the WF)
3. Create a rulebook with the run_workflow_job template and the job_args to populate that variable
4. Trigger the rulebook

Actual results

If the survey is set to required, it fails saying that variable is needed to start

2023-11-27 16:00:40,040 - ansible_rulebook.rule_set_runner - INFO - action args: {'name': 'Automated Response Node Exporter Down EDA', 'organization': 'Security', 'job_args': {'extra_vars': {'vm_name': 'rhel8.shadowman.dev'}}}

2023-11-27 16:00:40,041 - ansible_rulebook.action.run_workflow_template - INFO - running workflow template: Automated Response Node Exporter Down EDA, organization: Security

2023-11-27 16:00:40,041 - ansible_rulebook.action.run_workflow_template - INFO - ruleset: Automatic Remediation from Alert Manager, rule Prometheus Node Exporter Down

2023-11-27 16:00:40,292 - ansible_rulebook.job_template_runner - WARNING - Workflow template Automated Response Node Exporter Down EDA does not accept limit, removing it

2023-11-27 16:00:40,292 - ansible_rulebook.job_template_runner - WARNING - Workflow template Automated Response Node Exporter Down EDA does not accept extra vars, removing it

2023-11-27 16:00:40,524 - ansible_rulebook.job_template_runner - ERROR - Error connecting to controller 400, message='Bad Request', url=URL('https://tower.shadowman.dev/api/v2/workflow_job_templates/612/launch/')

2023-11-27 16:00:40,524 - ansible_rulebook.job_template_runner - ERROR - Error {'variables_needed_to_start': ["'vm_name' value missing"]}

Expected results

The job should successfully run without setting prompt on launch for extra vars since the survey exists

Additional information

No response

[mkanoor]

@adworjan In our testing with 4.4.0 we noticed the following error if we send extra vars into a launch of workflow template

{"extra_vars":["Variables abc are not allowed on launch. Check the Prompt on Launch setting on the Workflow Job Template to include Extra Variables."]}

Along with a bad request from the controller.

So if there is no prompt on launch enabled in the Workflow Template we won't be able to start the job.

This is different from a Job Template, a Job Template launch seems to accept extra vars even if the Prompt on Launch is not enabled.

[adworjan]

This is only an issue when using the run_workflow_template action. I can call the API directly without having prompt on launch checked (but with the survey enabled). Your test seems to not have a survey with variable abc set

[mkanoor]

When we get an event payload there is all kinds of data in it and we don't validate any of the data to see if it matches the survey spec types.

{'variables_needed_to_start': ["Value 23 for 'abc' expected to be an integer."], 'extra_vars': ['Variables hello, ansible_eda are not allowed on launch. Check the Prompt on Launch setting on the Workflow Job Template to include Extra Variables.']}

So even if we checked the prompt_on_launch for variables and survey enabled we are still going to see launch failures because of the kinds of data that we might get in the event payload. Currently EDA doesn't do anything with survey specs, we don't fetch it we don't validate if the extra vars match the name/type specified in the survey spec.

[adworjan]

The issue I have isn't with the survey spec type (I'm using the text type). I can send an API call to that workflow launch endpoint and the job runs with the extra var passed in. If I use the run_workflow_template it fails UNLESS I have prompt on launch checked. I'm still populating the exact same variable in the survey, but it only populates with prompt on launch checked. Every API call I make to controller works with JUST the survey enabled. This works fine in the run_job_template action, so the issue appears to be in the run_workflow_template action itself.

[mkanoor]

The run_workflow_template action is intentionally removing the extra_vars if the prompt on launch for extra vars is not set. The run_job_template doesn't do that. The job template launch on the controller is not as restrictive as is the case with workflow template launch. Even in the run_job_template action we have been recommending users enable the prompt on launch so you can see what is the extra vars that were sent up. https://github.com/ansible/ansible-rulebook/pull/566

[adworjan]

Can we remove that restriction? It would make sense to provide consistency between both actions (and just like with a Job Template, I might not need the ansible_eda data, I might just want a single variable). I purposely only use surveys so users don't have to know what variables to populate AND so I don't need a separate workflow for EDA vs regular usage.

[mkanoor]

That restriction is in place because of the controller restrictions on workflow templates launch procedure. Without the prompt on launch the run_workflow_template will still fail because of the fact that we are always sending the matching events under the ansible_eda key. This is the extra_vars that would get sent up

{'extra_vars': {'hello': 'Fred', 'abc': 23, 'ansible_eda': {'ruleset': 'Test run workflow templates', 'rule': 'Run workflow template', 'event': {'i': 1, 'meta': {'source': {'name': 'range', 'type': 'range'}, 'received_at': '2023-11-27T19:27:04.946004Z', 'uuid': '928b170e-14d3-4c3c-ba2c-b01c54620ae1'}}}}}

The ansible_eda is an object with ruleset/rule/event. This object cannot be represented in the Survey Spec, if you represented it as a string you would get a different error

This is the controller error for that

Error {'variables_needed_to_start': ["Value OrderedDict([('ruleset', 'Test run workflow templates'), ('rule', 'Run workflow template'), ('event', OrderedDict([('i', 1), ('meta', OrderedDict([('source', OrderedDict([('name', 'range'), ('type', 'range')])), ('received_at', '2023-11-27T19:27:04.946004Z'), ('uuid', '928b170e-14d3-4c3c-ba2c-b01c54620ae1')]))]))]) for 'ansible_eda' expected to be a string."]}

So for this to work we would have to remove sending the event data in the extra vars, today it is being sent unconditionally.

If the ansible_eda is not defined in the survey spec. the error changes to

ERROR - Error {'extra_vars': ['Variables ansible_eda are not allowed on launch. Check the Prompt on Launch setting on the Workflow Job Template to include Extra Variables.']}

[goldyfruit]

This is different from a Job Template, a Job Template launch seems to accept extra vars even if the Prompt on Launch is not enabled.

Hi there,

"Job Template" doesn't receive as well the extra vars if Prompt on Launch is not checked, just spent the last hour trying to figure out why extra variables were not passed to the job template (as well as ansible_eda).

Enabling Prompt on Launch made my job template receives the extra vars:

{
  "which_user": "Gaetan",
  "ansible_eda": {
    "ruleset": "Demo rules with websocket as source",
    "rule": "Run job template",
    "event": {
      "type": "ovos-skill-personal.OpenVoiceOS:WhoMadeYou.intent",
      "data": {
        "utterances": [
          "who made you"
        ],
        "lang": "en-us",
        "utterance": "who made you"
      },
      "context": {
        "session": {
          "active_skills": [
            [
              "ovos-skill-personal.OpenVoiceOS",
              1702337196.0405762
            ]
          ],
          "utterance_states": {},
          "session_id": "default",
          "lang": "en-us",
          "context": {
            "timeout": 120,
            "frame_stack": []
          },
          "site_id": "unknown",
          "pipeline": [
            "converse",
            "padatious_high",
            "adapt",
            "common_qa",
            "fallback_high",
            "padatious_medium",
            "fallback_medium",
            "padatious_low",
            "fallback_low"
          ]
        },
        "lang": "en-us",
        "skill_id": "ovos-skill-personal.OpenVoiceOS"
      },
      "meta": {
        "source": {
          "name": "websocket",
          "type": "smartgic.eda.websocket"
        },
        "received_at": "2023-12-11T23:26:36.071602Z",
        "uuid": "05181bba-384f-4d33-bb2c-89eab804f0f1"
      }
    }
  }
}

[mkanoor]

This is different from a Job Template, a Job Template launch seems to accept extra vars even if the Prompt on Launch is not enabled.

Hi there,

"Job Template" doesn't receive as well the extra vars if Prompt on Launch is not checked, just spent the last hour trying to figure out why extra variables were not passed to the job template (as well as ansible_eda).

Enabling Prompt on Launch made my job template receives the extra vars:

{
  "which_user": "Gaetan",
  "ansible_eda": {
    "ruleset": "Demo rules with websocket as source",
    "rule": "Run job template",
    "event": {
      "type": "ovos-skill-personal.OpenVoiceOS:WhoMadeYou.intent",
      "data": {
        "utterances": [
          "who made you"
        ],
        "lang": "en-us",
        "utterance": "who made you"
      },
      "context": {
        "session": {
          "active_skills": [
            [
              "ovos-skill-personal.OpenVoiceOS",
              1702337196.0405762
            ]
          ],
          "utterance_states": {},
          "session_id": "default",
          "lang": "en-us",
          "context": {
            "timeout": 120,
            "frame_stack": []
          },
          "site_id": "unknown",
          "pipeline": [
            "converse",
            "padatious_high",
            "adapt",
            "common_qa",
            "fallback_high",
            "padatious_medium",
            "fallback_medium",
            "padatious_low",
            "fallback_low"
          ]
        },
        "lang": "en-us",
        "skill_id": "ovos-skill-personal.OpenVoiceOS"
      },
      "meta": {
        "source": {
          "name": "websocket",
          "type": "smartgic.eda.websocket"
        },
        "received_at": "2023-12-11T23:26:36.071602Z",
        "uuid": "05181bba-384f-4d33-bb2c-89eab804f0f1"
      }
    }
  }
}

@goldyfruit What I meant was Job Template launch doesn't fail if the "Prompt on Launch" is missing, it just ignores the extra vars and keeps going. In the case of Workflow Template the launch fails if we send it extra vars when Prompt on Launch is not defined.

[goldyfruit]

@goldyfruit What I meant was Job Template launch doesn't fail if the "Prompt on Launch" is missing, it just ignores the extra vars and keeps going. In the case of Workflow Template the launch fails if we send it extra vars when Prompt on Launch is not defined.

Thanks for the clarification.