Default namespace for workflows

[Trollgeir]

Is there a specific reason that the default namespace for a Workflow object doesn't inherit from WorkflowService?

[flaviuvadan]

Hello again @Trollgeir! By design a workflow service can operate independently of a workflow object. The service takes a namespace and offers specific operations for that namespace, such as resuming a workflow, deleting, etc. Some users have argo installed in multiple namespaces, so they can use a service for performing operations within that namespace, along with submitting workflows to that namespace. However, a workflow service also hosts a token and a domain where operations can be applied, such as workflow submission. Therefore, a workflow service can be used to, technically, submit a workflow to any namespace, provided that a namespace is specified. For instance, I can have a ws = WorkflowService(...) and delete a workflow from namespace A but then I can also create a w = Workflow(..., service=ws) and submit the workflow to namespace B via w.create(namespace='B'). Currently, the workflow has a condition upon creation to check whether a namespace was provided on the create(...) call and fall back on the Workflow.namespace field otherwise, which provides additional flexibility.

One problem that I noticed, as you brought up the question (thank you!), is that the overall namespace on the service, provided through __init__ is not even used by the service inside independent methods as the methods rely on the namespace provided to them. I believe this is where the confusion comes from... you may have passed a namespace to the service and expected that workflow to be submitted to that namespace by virtue of submission through the service. What do you think is a better design here? Perhaps we can remove the namespace from the workflow service __init__ and make it a general workflow orchestration service used for submission, listing, etc? Or, should we make the service take in a global namespace, as it does now, and make all calls use that global namespace? This prevent the workflow from setting a different namespace upon submission, so all workflows will go to the service namespace. Thoughts?

[Trollgeir]

... you may have passed a namespace to the service and expected that workflow to be submitted to that namespace by virtue of submission through the service. What do you think is a better design here?

Yeah that was my expectation. Right now I feel like this is a gotcha-scenario,

Removing it as an argument for the WorkflowService would eliminate the gotcha, which I like better than the way things are now.

I think Workflow shouldn't hold a namespace member, as it should either be decided explicitly upon creation, or default to namespace defined in the WorkflowService. The reason for this is that I see a workflow as a blueprint which can be instantiated anywhere, and defining specifically where it should be instantiated shouldn't be a part of the workflow object - that's up to the service.

[flaviuvadan]

After some deliberation, I realized that what you are describing @Trollgeir is actually way more consistent with the types of Argo installations users can have. In addition, it's more consistent with the per-namespace authentication model that can be performed if one uses a K8S SA token to auth for workflow submission. I think we should go ahead and remove namespace specifications from the workflow and rely on specifying those, exclusively, on the service. This will be a breaking change, which I think we should release with a 4.0.0 release. The complexity of the change is small but the impact is high. Mind if I mark this for a 4.0.0 release? 🙂