optimus icon indicating copy to clipboard operation
optimus copied to clipboard

Add api to get compiled job specs for a given schedule time

Open Mryashbhardwaj opened this issue 2 years ago • 0 comments

As part of this card, we expect a new api for fetching compiled job specs instead of the current approach where we have a side effect of registering the job run.

Scope:

  1. execution_time macro to be updated with the job_run_start time that is being captured with notify_event api call.
  2. ~~current register_instance api call should n't be used and should be replaced with this new endpoint.~~
  3. ~~drop the old job_run and instance tables.~~

Changes in the PR https://github.com/odpf/optimus/pull/435 :

  1. renamed the then "job_run" table to "job_run_old", and deprecate its use
  2. ~~deprecate "instance" table~~

Earlier :

  • RegisterInstance api used to register an job run instance in the "instance" and the then "job_run" table Now :
  • RegisterJobEvent api is used to register a job start in the new "job_run" table.
  • JobRunInput api is used to fetch compiled job assets.
    • This uses the new "job_run" table to fetch computed macro values

The compilation logic is not altered

Mryashbhardwaj avatar Jun 27 '22 06:06 Mryashbhardwaj