Hooks defined in Workflow Controller `workflowTemplates` cause every workflow to fail

[bencompton]

Checklist

• [x] Double-checked my configuration.
• [x] Tested using the latest version.
• [x] Used the Emissary executor.

Summary

What happened/what you expected to happen?

Any hook defined in the Workflow Controller ConfigMap under workflowTemplates causes workflows to fail with Workflow Operation Error and the error showing in the Workflow Controller logs is Unable to set ExecWorkflow" error=merging an object in json but data type is not struct, instead is: map.

What I expect to happen is every workflow that runs executes the hook defined in the Workflow Controller ConfigMap under workflowTemplates. This does work as expected with onExit instead of a hook, but my use case requires the more advanced functionality that hook and expression enables.

What version are you running?

3.3.9

Diagnostics

Paste the smallest workflow that reproduces the bug. We must be able to run the workflow.

    # Workflow Controller ConfigMap

    workflowDefaults:
      spec:
        hooks:
          exit:
            template: hook-test
        templates:
          - name: hook-test
            container:
              image: alpine:3.6
              command: [sh, -c]
              args: ["echo \"hook test\""] 

# Logs from the workflow controller:
time="2022-08-17T16:31:11.486Z" level=info msg="Processing workflow" namespace=argo-workflows workflow=workflow-hook-test-7sjn9
25
time="2022-08-17T16:31:11.486Z" level=info msg="Updated phase Running -> Error" namespace=argo-workflows workflow=workflow-hook-test-7sjn9
24
time="2022-08-17T16:31:11.486Z" level=info msg="Updated message  -> merging an object in json but data type is not struct, instead is: map" namespace=argo-workflows workflow=workflow-hook-test-7sjn9
23
time="2022-08-17T16:31:11.486Z" level=info msg="Marking workflow completed" namespace=argo-workflows workflow=workflow-hook-test-7sjn9
22
time="2022-08-17T16:31:11.486Z" level=info msg="Marking workflow as pending archiving" namespace=argo-workflows workflow=workflow-hook-test-7sjn9
21
time="2022-08-17T16:31:11.486Z" level=error msg="Unable to set ExecWorkflow" error="merging an object in json but data type is not struct, instead is: map" namespace=argo-workflows workflow=workflow-hook-test-7sjn9
20
time="2022-08-17T16:31:11.486Z" level=info msg="Checking daemoned children of " namespace=argo-workflows workflow=workflow-hook-test-7sjn9

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.