graph-notebook
graph-notebook copied to clipboard
Allow %load_ids magic to expose load status for each bulk loader job
Is your feature request related to a problem? Please describe.
The %load_ids
magic currently fetches a full list of all bulk load IDs that were previously submitted to a writer instance. However, to fetch the status of each load job, a user would then have to use individual calls to %load_status
to provide any further details for each job. If a user is submitting multiple bulk load jobs (particularly when they are queuing multiple jobs), there's no easy way to see which jobs are queued and which ones are executing/completed.
Describe the solution you'd like
Add a --details
flag to %load_ids
that results in a table of all bulk load jobs that have been submitted and the status for each (ex: LOAD_COMPETED
, LOAD_QUEUED
, LOAD_IN_PROGRESS
, etc.). Potentially also provide a --limit X
parameter to only return the status for the last X bulk load jobs, as some users may have many hundreds of jobs that have previously been submitted.
Describe alternatives you've considered
An alternative would be to provide an example of using a notebook variables to take the output of %load_ids
and pipe that into multiple cells of %load_status
, but that would be cumbersome if one needed to review status for dozens of issued bulk load jobs.
Additional context N/A