Moving a Job from one namespace to another should be seamless without any manual intervention

[sravankorumilli]

When a job is moved from one namespace to another, it should be seamless, the job needs to be dereferenced from the old namespace and referenced in the new namespace. Even the compiled job spec to be dereferenced from the scheduler.

[kushsharma]

What exactly is broken in current behaviour? I thought users just create a new job in new namespace and delete the existing one?

[Mryashbhardwaj]

Analysis:

Issue:

• Currently there is no flow to identify the movements of jobs across namespaces.
• When moved, old job is deleted and new ones are created.
  • Due to this we lose old job from Optimus perspective.
  • This does not impact Airflow, but this is a whole new job for Optimus.
  • This can create problems in referring old job by job Id. This shall impact job_run instances
    • Impact area job run monitoring

Approach to Fix:

1. Do this as part of the job deployments.
2. Fetch all jobs per "namespaces to be deployed" from Optimus server.
3. Check the removed jobs, and recursively check if they exist in some other folder now.
4. Determine job movements across namespace.
5. Then send namespace change information to Optimus in the deploy command.
6. This will change the namespace association in the db and will do deployment accordingly.
7. New flow needs to be added to handle the migrated jobs differently, just like created/deleted/modified jobs.
8. In the new flow, only the namespace shall be updated in jobSpec and file location shall be updated on the scheduler.
9. DAGS will need to be deleted and created in new folders just like in case of job deletion / creation.

Note:

• We have support for selective namespaces deployments, need to check the interplay of this with namespace migration
• It is debatable, should the job_runs be associated with old namespaces or new ones after the migration

[sravankorumilli]

Agreed @Mryashbhardwaj we need to have a move implementation in the deploy all api

[Mryashbhardwaj]

sure, we can pick this up in upcoming sprints.