jupyter-scheduler
jupyter-scheduler copied to clipboard
jupyter-server
Server-side support for event-driven notebook execution
Adds server-side support for event-driven notebook execution, fixes #406 (for UI see #414).
System now defines EventTypes and Events. EventType is specific condition identified by a name and a set of parameters. Event is a unique occurrence of the certain event type.
Event types can be defined through RuntimeEnvironment.event_types[]. Event occurrences are supposed to originate from outside of JupyterLab, from the broader environment where it operates. When an event of a certain type occurs, all job definitions associated with the event type (JobDefinition.events[] has an item with matching name) are supposed to be queried. This PR does not provide a mechanism to kick-off the event occurrence and does not manage querying of matching job definitions, it just provides models and handlers to support the process.
Changes
Non-breaking REST API changes:
POST /job-definition
Existing endpoint to create or update a job definition. Add new optional parameter events that contains array of event types that trigger job definition execution
Request schema:
POST /job-definition
{
...
// Optional; array of event types that trigger job definition execution
"events": Array<{ "name": string, "parameters": object }>
}
POST /job
Existing endpoint to create or update a job. Add optional parameter triggered_by that contains an event that triggered the job execution
Request schema:
POST /job
{
...
// Optional; event that triggered this job exectution
"create_event": {
"event_id": string,
"event_type": string,
"parameters": object
}
}
Changes to existing models
RuntimeEnvironment:
class RuntimeEnvironment(BaseModel):
...
event_types: Optional[List[EventType]]
CreateJob, DescribeJob:
class CreateJob(BaseModel):
...
create_event: Optional[Event]
CreateJobDefinition, DescribeJobDefinition, UpdateJobDefinition:
...
events: List[EventType] = []
New models
class EventType(BaseModel):
name: str
parameters: Dict[str, Any]
class Event(BaseModel):
event_id: str
event_type: str
parameters: Dict[str, Any]
Changes to existing models
ORM changes
class Job(CommonColumns, Base):
...
create_event = Column(JsonType(1024), nullable=True)
class JobDefinition(CommonColumns, Base):
...
events = Column(JsonType(1024), default=list)
